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ABSTRACT 

This report was prepared to provide information on 
school evaluation practices to the education community and the 
general public. It is intended to assist individuals in selecting 
school evaluation models that assess the dimensions and components of 
a school that are considered most relevant and most related to school 
improvement. Work on the report itself was preceded by an intensive 
search for extant school evaluation models from various sources. In 
all, 51 models were received from national organizations, school 
districts, state departments of education, and various agencies with 
an interest in evaluation. Using these models, research associates 
developed profiles based on key characteristics, and models were then 
classified into the typology/evaluation matrices that are presented 
in this report. After the user identifies the appropriate matrix, 
types, purposes, processes, and levels, focus characteristics are 
used to find models with the components the user needs. A final step 
is obtaining detailed descriptions that can be adopted or adapted for 
the user's purposes. Four charts summarize model characteristics. 
Four appendixes give profiles of national, regional, state, and local 
education agency models. (Contains 6 references.) (SLD) 
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L INTRODUCTION 



Consumer reports typically provide consumers with independent evaluations of 
competing products or services, such as makes of a type of automobile or alternative 
life insurance policies. This consumer report is aimed at educational decision makers 
who share in the responsibility to select and/or to design models for evaluating 
schools. Of course, evaluating school evaluation models is not quite the same as 
evaluating such things as automobiles or washing machines. Therefore, to assist the 
reader the introductory section discusses the purposes of the study, contextual 
considerations, and methods used. These issues, although relatively straightforward, 
are important for reader background to avoid unwarranted inferences about how the 
study was conducted and ratings assigned to the models evaluated. 

Purposes of the Report 

This report was prepared to provide information on school evaluation practices to the 
education community and the general public. The information provided herein should 
be of interest and use to individuals employed in school systems or schools that are 
systematically evaluated as well as to individuals who carry out evaluations of 
schools. Such evaluations may occur under the auspices of regional accreditation 
agencies, state departments of education, school districts, or individual schools 
(campuses) both public and private. It is also assumed that the general public will 
have an interest in this document. 

The report is intended to assist individuals in selecting school evaluation models that 
assess those dimensions and components of a school that are considered most 
relevant and whose assessment would contribute to school improvement. The report 
is also intended to contribute to the quality of information included in self-studies 
prepared for submission to internal or external evaluators. The models reviewed 
were not all-inclusive of school evaluation models, because the originators of the 
models had the option of submitting or not submitting their models for inclusion. 
Nevertheless, the models reviewed are descriptive of models used at the various 
levels by different organizations and are informative in identifying components of a 
more "ideal" school evaluation modeL To that end it is anticipated that this report 
will encourage and assist agencies and organizations that conduct school evaluations 
to improve and modify their evaluation practices. 

Contextual Considerations 

At the time this report was being prepared, changes in school organization and 
administration were occurring at an increasing rate in the United States. These 
changes are having an impact on school evaluation practices. Nevertheless, it was 
considered important to assess current practices in school evaluation to establish a 
"state-of-the-art" baseline. The knowledge in most areas of inquiry is advanced by 



building on the present base, and there was no reason to believe that school 
evaluation would be an exception. 

A major consideration in assessing the value of current school evaluation models was 
the importance given to the 'context" of the education environment (e.g., general 
climate for reform, demographic changes, socioeconomic status, community support 
and involvement, personnel evaluation practices, decision-making processes, etc.) and 
its inevitable impact on school effectiveness. This consideration required evaluating 
models not onl> against their own stated objectives but, also to the extent possible, 
against their provisions for accommodating a wider range of contexts. Also, along 
with the classifications and ratings given and the unique strengths identified in the 
models reviewed, recommendations were made for strengthening the contributions 
of school evaluation models to educational reform and school improvement. 

Methods Used 

Work on this report was preceded by an intensive nationwide search for extant school 
evaluation models from various sources. Initial contacts were made by letter, with 
a telephone follow-up as needed, to national organizations and associations, all 
recognized regional accrediting associations, state departments of education, 
individual school districts, private school organizations, specialized agencies, and for- 
profit organizations involved in merchandising evaluation materials. A request was 
made for copies of school evaluation models used by these associations, agencies, and 
organizations, if available, and for the names of school districts or schools that were 
considered to have worthwhile evaluation models that could then be obtained and 
included in our overall evaluation. 

These initial contacts were often followed up by a phone call to clarify the request or 
to inquire about why we had not received the requested information. Many of our 
contacts claimed not to have school evaluation models or were in the process of 
developing one (e.g., Florida), Others did have an extant model available for review 
but were in the process of developing anew or optional model. 

Upon receipt of the models (51 in all), research associates in the Center developed 
profiles based on key characteristics for each of the models (see Appendices). These 
profiles, along with the actual model documents themselves and a specially designed 
classification/rating form that conformed to the evaluation system developed (see 
Section II) were then used by consultant evaluators to carry out an in-depth 
evaluation of the models. The evaluators worked independently to classify and rate 
the models, but then came together to compare results. Check marks were used to 
classify each model by type and characteristics. A 1-5 rating system was used to rate 
models (see bottom of matrix for rating definitions) in relation to the evaluation 
criteria applied. Co-rater reliability was consistently high. However, the reader must 
keep in mind that all evaluation includes some element of subjectivity and that the 
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ratings given were based on judgments by the evaluator consultants. Initial 
classifications and ratings were shared with the model originators for verification and 
feedback. 

The classifications were based on the use of selected model characteristics considered 
significant (e.g., type, intended use, foci, procedures), and the ratings were 
established through the application of evaluation criteria that are related to the 
various dimensions of the Context, Input, Process, Product (CIPP) Model 
(Stufflebeam, 1983); a set of accountability strands relating to quality; and the 
various dimensions of the Joint Committee Standards for Evaluations of Educational 
Programs, Projects, a n d Materials (Joint Committee, 1981). The reader is referred 
to the GUIDE TO TYPOLOGY/EVALUATION MATRIX section in this report to 
obtain an understanding of the categories that contain the various characteristics and 
criteria used in classifying and evaluating the school evaluation models that are 
displayed in the typology/evaluation matrices included in this report. Distinct display 
matrices are included for school evaluation models developed or distributed by 
national organizations, associations, or agencies; for models developed and utilized 
by regional accredition associations; for those developed and implemented by state 
education agencies; and for those models developed by school districts, schools, or 
consortia of schools or districts. 

Before turning to the specific content of the typology/evaluation matrix, some 
comments about the overall process are in order. Evaluation is a process that 
involves judgment and subjectivity, and documents such as descriptions of evaluation 
'models may be misinterpreted. When the met&evaluation of a model was completed, 
the principal authority or developer of the model was provided with a draft of the 
classifications and ratings given to the model along with the profile developed and the 
guide to the categories and criteria used. A cover letter requested a review of the 
information in the matrix and the profile. This review provided an opportunity to 
clarify any misinterpretations and to correct any factual errors based on feedback 
from developers. Of course, appropriate documentation was required for any 
suggested changes. As required of any independent consumer report, the consultants 
retained independence in providing judgments of the models. In essence, this review 
provided a verification check and, with the implementation of this process, it is 
presumed that the models are described and represented accurately throughout. 

Readers should not infer that the models reviewed had been field tested or previously 
evaluated by their originators or anyone else. From the model descriptions, there 
was no way of determining whether or not models had been field tested or subjected 
to metaevaluation. 
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II. GUIDE TO TYPOLOGY/EVALUATION MATRIX 

This section describes the typology/evaluation matrix. The content of the matrix is 
presented, followed by an overview of the rating process. Finally, there is a 
discussion of how the reader can use the consumer report most effectively. 

Content of the Typology/Evaluation Matrix 

The various categories and their respective characteristics or evaluation criteria used 
in the typology/evaluation matrix display (see page 5) were an outgrowth of an 
attempt to consider all significant aspects of schooling that should be taken into 
account when evaluating the overall quality of a specific school- It is recognized that 
some of the categories used for characterizing and evaluating the various models go 
beyond the traditional considerations normally dealt with in evaluating schools. 
Nonetheless, it was felt that the inclusion of these additional categories was useful 
to the identification of key characteristics and unique strengths in current models in 
light of new emerging perspectives (e.g., the inclusion of more qualitative data along 
with quantitative data). 

The work of preparing this consumer report was one of metaevaluation-that is, the 
process of evaluating an evaluation. As metaevaluators, the consultants took into 
account the stated purposes of the models reviewed and the extent to which the 
models were designed to attain their intended outcomes. But the metaevaluation was 
broadened, and consultants also examined the models against professional standards 
of sound evaluation and their provisions for evaluation of student outcomes, 
educational inputs and processes, school environment, and other school variables. 
Consultants exercised the prerogative of examining the models on the bases of 
emerging considerations in school evaluation, such as those relating to quality issues 
as well as student outcomes. 

This section of the report, then, explicates the meaning of the various categories in 
the typology/evaluation matrix applied in conducting a comprehensive analysis of 
evaluation models. The matrix organizes the information by model The categories 
used to characterize the various models reflect the best attempt to inform the 
profession (and any other interested parties) about relevant variables that are or are 
not considered in each model. This best attempt includes the identification of 
"generic" characteristics essential to all models, yet synthesized in such a manner 
that the information is concise and manageable. As advances are made in school 
evaluation, undoubtedly the matrix will require change or adjustment in order to 
maximize its usefulness. 
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Types/Purposes 

School evaluation is a broad and extensive topic, and the purposes for conducting 
school evaluation, both explicitly and implicitly, are many and varied. For example, 
a single, broad purpose for all school evaluation is for school compliance with state 
and federal regulations. However, such a perspective was too limited for this report, 
so it was useful to apply the three major purposes of school evaluation identified by 
Richards (1988) as the basis for the types/purposes of the models. These purposes 
provide a broad perspective on school evaluation. They are described as follows: 

o C=Compliance monitoring: Used to determine if schools are operating in 
accordance with some predetermined, externally imposed standard(s) emphasizing 
"inputs" (i.e., resources available) rather than performance, or are being monitored 
against internal policies. 

o D=Diagnostic monitoring: Focuses on student improvement and, therefore, mostly 
relies on criterion-referenced pre- and posttesting patterns. Program monitoring is 
primarily formative in nature and focuses on how assessed student needs are being 
addressed. 

o P=Performance monitoring: Focuses primarily on the results of norm-referenced 
achievement tests given usually at the 4th, 8th, and 10th grade levels. It is 
summative and intended to enable schools and districts to be compared currently 
and longitudinally. 

Process 

The three classifications listed under this category serve to indicate if the school 
evaluation model calls for a self-study on the part of the school staff, requires a 
visitation by an outside evaluation team or consultant(s), or simply omits the external 
visitation in favor of a desk audit by some official who receives the required 
documentation and makes judgments based on the information provided. Any one 
or a combination of these three processes may apply to a given model. The processes 
are indicated by SS for self-study, V for visitation, and DA for desk audit. 

Levels Focus 

This is a straightforward classification of a school evaluation model relating to the 
school level for which it is designed, i.e., elementary (E), middle school (M), high 
school (H). In this column, some models have "Disk" indicated under the EMH. This 
means that the model applies to all levels in the district. If Dist. is not indicated, it 
means that the submodels, so to speak, for the different levels differ somewhat across 
levels. In the state models matrix, Hawaii has "state" below the EMH, meaning that 
the same applies to all levels and the entire state of Hawaii is considered a district. 

6 
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Intended Focus 



School evaluation data can be scrutinized from two perspectives: formative and 
summative. These perspectives focus on the intended use of the evaluation results. 
Formative evaluation focuses on insights the data may provide for school 
improvement. Summative evaluation focuses on decisions about program 
effectiveness and, in some situations, about whether or not the program should 
continue. 

o Form=Formative evaluation: Provides data for planning specific program 
improvements. 

o Sum=Summative evaluation: Provides data and evidence on how well a program 
meets the needs of the client group and whether or not the effort was efficient and 
effective. 

Data Focus 

The methods and the instruments employed in an evaluation will determine the type 
of data collected, analyzed, and eventually used. For information to be useful for 
school accountability and improvement, an evaluation should contain a balance of 
quantitative and qualitative information. 

o Quantitative information refers to types of data generally gleaned from rating 
scales, aggregated and disaggregated test scores, selected response questionnaires, 
semantic differentials, etc. which are pertinent to the program or system's mission 
and objectives. 

o Qualitative information generally refers to information gleaned from focused 
interviews, climate studies, anecdotal records, independent observer input, key 
informants, portfolios, case studies, hearings, site visits, etc., that provide insights 
into a school's morale, psychological impact on students and teachers, community 
perception and support, and curricular state of the ait with an emphasis on 
improvement. Qualitative data do not lend themselves easily to statistical 
treatment and focus on individual cases. In contrast to quantitative data, which 
tend to be highly objective, qualitative data include some degree of subjectivity. 

Planning/Decision Making 

Planning/decision making is important in any evaluation, so it was decided to use the 
CIPP model (Stufflebeam, 1983) with its four components. The CIPP model links 
process and product within programs being evaluated and therefore served the 
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purpose of this study well. It applies four types of evaluation to the process of 
planning and decision making: 

o C=Context evaluation: Identifies and describes the actual and desired conditions 
surrounding a program. Current program status, unmet student needs, 
opportunities for strengthening educational services, and problems preventing 
needs from being met and opportunities from being pursued are identified. 

o I=Input evaluation: Seeks useful information for determining how to allocate 
limited resources to address assessed student needs and meet established 
objectives. Alternative program strategies are evaluated for responsiveness to 
assessed student needs and for cost and compatibility with existing district 
structures. 

o Pl=Process evaluation: Detects differences between what was planned to occur and 
what is actually occurring; alters courses of action to meet the intent of the original 
design or to correct a deficient design; documents events and procedures by 
recording what is occurring in relation to the planned course of action. 

o P2=Product evaluation: Measures intended and unintended program and school 
outcomes and interprets them in consideration of assessed student needs and 
program objectives and the information collected on program inputs and processes. 

Accountability Strands 

When evaluating schools, certain assumptions must be made that explicate the 
underlying values to be upheld by the schools. These values relate to "accountability 
strands" (Stufflebeam, 1991), the components by which schools can be held 
accountable to their stakeholders. The four strands are described below: 

o G=Growth (developmental needs): Focuses on whether or not the educational plan 
was derived from a sound needs assessment that addressed all developmental needs 
and whether or not all students are achieving at acceptable levels in the 
developmental areas identified (i.e., intellectual, emotional, social, vocational, 
moral, aesthetic, and physical/health development). 

o E=Equity (equal opportunities): Looks at disaggregated data to determine whether 
or not all classifications of students have access to and participate equitably in the 
full range of services offered by the school and determines the extent of equitable 
impact of school plans and activities on the performance of all segments of the 
student population. 
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o Fl=Feasibility: Determines whether or not the school's priorities are clear, 
realistic, and appropriate; if plans are operational and realistic; and if resources are 
used cost effectively. 

o Ex=Excellence: Uses qualitative and quantitative data to assess the school and the 
community to determine whether or not there is a climate conducive to learning 
and whether excellence in teaching and student performance is emphasized and 
rewarded. 

Metaevaluation Standards 

When conducting metaevaluation, the evaluations (in this study of evaluation models) 
should be judged by an appropriate set of professional standards. The definitive work 
in this area, Standards for Evaluations of Educational Programs, Projects and 
Materials (Joint Committee, 1981), was done over a decade ago by the Joint 
Committee on Standards for Educational Evaluation. There are 30 standards 
distributed over four categories. To include all 30 standards in the 
typology/evaluation matrix would be too cumbersome, so the four categories are 
included in the matrix. These four categories and their respective standards provide 
a working philosophy of evaluation as well as principles that should guide evaluation 
efforts. The information in the matrix is provided by category, although that 
information reflects a consideration of all the standards in a category as they applied 
to a specific model. 

Any reader interested in a detailed description of the Standards is referred to the 
McGraw-Hill publication. The categories have varying numbers of standards. For 
the reader's information, descriptors of standards within each category are given 
following the definitions below. 

o U=Utility standards: Ensure that an evaluation will serve the practical 
information needs of given audiences. 

1. Audience Identification 

2. Evaluator Credibility 

3. Information Scope and Selection 

4. Variational Interpretation 

5. Report Clarity 

6. Report Dissemination 

7. Report Timeliness 

8. Evaluation Impact 
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o F2=Feasibility standards: Ensure that an evaluation will be realistic, prudent, 
diplomatic, and frugal. 

1. Practical Procedures 

2. Political Viability 

3. Cost Effectiveness 

o P=Propriety standards: Ensure that an evaluation will be conducted legally, 
ethically, and with due regard for the welfare of those involved in the evaluation. 

1. Formal Obligation 

2. Conflict of Interest 

3. Full and Frank Disclosure 

4. Public's Right to Know 

5. Rights of Human Subjects 

6. Human Interactions 

7. Balanced Reporting 

8. Fiscal Responsibility 

o A=Accuracy standards: Ensure that an evaluation will reveal and convey 
technically adequate information about the features of the object being studied that 
determine its worth or merit. 

1. Object Identification 

2. Context Analysis 

3. Described Purposes and Procedures 

4. Defensible Information Sources 

5. Valid Measurement 

6. Reliable Measurement 

7. Systematic Data Control 

8. Analysis of Quantitative Information 

9. Analysis of Qualitative Information 

10. Justified Conclusions 

11. Objective Reporting 

System Focus 

This matrix category refers to the various levels of interaction and linkage required 
to perform adequate evaluations of schools and school systems. That is, even though 
evaluation models should allow for linkages in data gathering and analysis for all 
components of the CIPP model, provisions should also be made to relate the school 
level evaluation to that being conducted districtwide and to any state-mandated 
evaluation. This dimension of "interactive linkages" is considered important in 
assessing the contribution and performance of each individual component to the 
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performance of the larger organization. The system focus column in the matrix 
represents the extent to which such functional interactive linkages are contained in 
the model. 



Unique Strengths 

The strengths of specific evaluation models are reflected in the ratings of the 
characteristics in the matrix. That is, a rating of 1 indicates a strength on that 
particular characteristic. Correspondingly, weaknesses of models are reflected in 
ratings of 4 on the other end of the scale. (Ratings of 3 or 5 may also be interpreted 
as indicative of weaknesses.) The seventeen characteristics rated in the matrix 
represent what might be called "generic" characteristics expected to be common to all 
school evaluation models. 

As the metaevaluations were conducted, it seemed appropriate and useful to 
recognize strengths in the school evaluation models that went beyond the generic 
characteristics and in essence were unique strengths of the models (hence, the title 
for the column). A unique strength is a strength found in one or more models, but 
not in all models. The inclusion of these strengths is part of an attempt to identify 
characteristics of what might be considered an ideal school evaluation model 

The converse should not be inferred; that is, that lack of a unique strength is a 
weakness. Weaknesses are indicated only in the ratingB according to the numerical 
code. Weaknesses should not be attributed to models solely on the basis of missing 
check marks or limited levels focus. For example, suppose a reader is interested in 
models that apply to all levels, K-12. If a specific model applies only to E and M 
levels, that should not be viewed as a model weakness, but only that the model is not 
appropriate for the reader's purpose. 

The list of unique strengths is based on the work related to indicators of quality 
conducted by Oakes (1989), the Special Study Panel on Education Indicators for the 
National Center for Educational Statistics (1991), and others. No claims are made 
that the list is all-inclusive, but it served well the purposes of this study. 
Undoubtedly, the list will be updated as the field of school evaluation develops. The 
list is coded as follows: 

A. Provisions are m«de for adequate training of those responsible for carrying out 
the evaluation (e.g., visiting team members and/or staff). 

B. The evaluation process involves all key stakeholders and calls for participatory 
decision making. 

C. Professional staff evaluation criteria and procedures are examined. 
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D. The degree to which demonstrated student learning is central to the 
professional staff evaluation process (a significant dimension of C. above) is 
considered. 

E. The evaluation encourages authentic assessment 1 of professional staff 
performance. 

F. The evaluation encourages authentic assessment of student achievement and 
development. 

G. The data collected encourage achievement on a broad range of high level 
outcomes. 

H. The evaluation considers faculty assumptions and expectations about learning 
abilities for all students. 

I. The level of top administrative advocacy and support for challenging 
curriculum and instruction is assessed. 

J. The degree of administrative and board support for professional risk taking 
and experimentation is assessed. 

K. The evaluation process and product encourage continuous improvement. 

L. The purposes and outcomes of staff development are explicitly assessed. 

M. The extent, focus, and quality of teacher participation in schoolwide decision 
making are evaluated. 

N. The gathering, analyses, and reporting of longitudinal gains are required. 

0. The systematic use of evaluation results to guide interventions is encouraged. 

P. The extent of external networking to support instruction at home and at school 
(e.g., parents, community agencies, business) is evaluated. 

Q. An evaluation is made of academic support programs (e.g., tutoring, after 
school remediation). 



1 The observation or documentation of actual use or application of knowledge, 
skills, or attitudes acquired (e.g., problem solving, hands-on application, etc,), 
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R. The evaluation examines opportunities for extracurricular enrichment (e.g., 
field trips, etc.). 



S. The program for student and teacher awards/recognition is assessed. 

T. The data/information collected and the required follow-up procedures will lead 
beyond school improvement to school restructuring, as appropriate. 

U. Evaluation is tied to efforts to improve schools at the community, district, 
state, and federal levels. 

V. The data gathered contribute to assessing progress toward achieving national 
goals. 

W. Disaggregated data analyses by gender, race, SES, etc., are required. 

Bases for Ratings 

Metaevaluation involves some considered judgment about the salient characteristics 
of the object being evaluated. In the matrix, 17 characteristics grouped under 6 
categories were rated on a 1 to 5 scale. As each model was evaluated, it was 
necessary to interpret not only the quantity of the information provided about a 
characteristic, but also the quality. The consultant evaluators discussed the 
characteristics of each model until any differences in ratings were resolved to the 
satisfaction of all. In general, the ratings were based as follows: 

Rating 1 (highest rating). More Than Meets 

The model required more than one source of information to verify conditions 
or practices within a school related to the characteristic, and the data were 
analyzed from more than one perspective. 

Rating 2. Meets 

The model required at least one source of information to verify conditions or 
practices within a school related to the characteristic and there was adequate 
data analysis. 

Rating 3. Partially Meets 

The model addressed the characteristic and at least one source of information 
related to the characteristic was identified to verify conditions or practices 
within a school, but the methods and/or data to be obtained were inadequate 
for providing the necessary information. 
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Rating 4. Does Not Meet 

The model addressed the characteristic, but no credible source of information 
about conditions or practices with a school was identified; or, if a source was 
identified, there were no appropriate analyses described that would provide 
the necessary information. 

Rating 5. Insufficient Information 

This rating was included to guard against an unwarranted negative rating 
when the information contained in the model description was inconclusive 
relative to the characteristic. 

To give readers an illustration of the ratings for a characteristic, consider the 
characteristic E (Equity) of the Accountability strands. A 1 rating indicates that 
student data are comprehensively disaggregated by gender, race, ethnicity, and 
socioeconomic status, and analyzed not only in relation to outcomes (e.g., test scores) 
but also in relation to curricular enrollment patterns, attendance, extracurricular 
activities, dropout rates, etc. A 2 rating (Meets) was given if a model called for 
disaggregated student performance data based on gender, race, ethnic background, 
and socioeconomic status. A 3 rating (Partially Meets) was assigned if a school was 
required only to disaggregate data on a limited basis, such as gender. A 4 rating 
(Does Not Meet) was given when disaggregated data were not required of a school; 
and* of course, a 5 was given if there was insufficient information about equity. 

A word of caution about using the ratings in the matrices-there may be a temptation 
to sum the ratings for the seventeen characteristics and select the model with the 
lowest sum or average rating. (The ratings are so ordered that the lower the value 
of the rating, the stronger the model is on the characteristic.) Computing a sum or 
average for the ratings is NOT appropriate. Although the ratings are 1 through 5, 
they are not on interval scales. In fact, rating 5 is not on an ordinal scale with 
ratings 1 through 4. Computing a sum or average, even if appropriate in terms of 
measurement, would assume equal importance for all characteristics. It would be 
very rare that such would be the case for any user's situation. Some characteristics 
will be essential, others of less importance to the user. There may be models that 
have high ratings but have requirements that cannot be met by a specific user. So 
the ratings are there to guide the user's decision making, and by themselves they 
comprise necessary but not sufficient information for deciding on one or more models. 

Walk-Through Example 

To demonstrate how the various models wer<" reviewed, a description of how the 
Colorado State Department of Education Model was classified and rated is presented. 
To begin, project staff developed a comprehensive profile of the model upon receipt. 
The consultants then applied the previously developed classification and rating 
system and a companion rating form based on the categories listed on the 
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Typology/Evaluation matrix. The model's profile and accompanying materials (e.g., 
manuals) were then analyzed, classified, and rated. The classifications and ratings 
given to the model were then transposed to the matrix of the state models. 

If one considers the TYPOLOGY/EVALUATION MATRIX FOR STATE MODELS (see 
page 28) and looks at the classifications and ratings given the Colorado State 
Department of Education model, one sees that it was found to encompass all three 
types of school evaluation-that is, the model obtains and provides data/information 
relating to compliance, diagnostic, and performance issues. The matrix show that the 
state requires a self-study and a desk audit and it also allows for an on-site visit by 
an external team of evaluators that relates to both accreditation and school 
improvements. The model addresses school evaluation at the elementary, middle 
school, and secondary school levels. 

The ratings for characteristics show that the model is both formative and summative 
in nature, with a greater emphasis on the former, and that it calls for both 
quantitative and qualitative data/information. The model is designed to be strong in 
evaluating the contextual and performance dimensions of schools while evaluating the 
inputs and procedural aspects of the schools. 

Further analysis showed that the model requires schools to develop educational plans 
based on sound needs assessments of learners, that it calls for schools to disaggregate 
student data so that determinations can be made about equal opportunity for all, and 
that this model evaluates schools'/school districts' priorities and plans to determine 
their feasibility and whether or not there id a climate in the school setting conducive 
to excellence. It was agreed that the model sufficiently addressed the four categories 
of the metaevaluation standards relating to utility, feasibility, and propriety, and that 
accuracy was partially met. The final characteristic rated shows that the model calls 
for adequate interactive linkages between and among individual schools, the district 
as a whole, the state department of education, and any regional accreditation activity 
that may be required. The last column in the matrix indicates that this model has 
some unique strengths, such as creating a press for authentic assessment of student 
achievement and development; encouraging continuous improvement through the 
gathering, analyses, and reporting of longitudinal gains; and calling for the 
disaggregation of data to permit an analysis of student performance by gender, race, 
SES, etc. 

The Colorado State Department of Education Model, then, is quite comprehensive and 
would be useful to anyone seeking an evaluation model with a broad array of 
information about schools. Other models, such as those from Indiana and Texas, are 
also comprehensive and deal in depth with compliance, diagnostic, and performance 
issues. Other state models have clear voids in their foci. For example, the 
Connecticut Department of Education recently completed work on a compliance 
model, a characteristic missing in the model reviewed herein and so indicated in the 
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matrix. Additional information about models is provided to readers in the model 
profiles of the appendices. 



Process for Using the Consumer Report 

Use of the consumer report is quite straightforward, but at this point it is appropriate 
to make some comments that may enhance user effectiveness as well as efficiency. 
Presumably, any potential user is someone identifying a school evaluation model that 
may be adopted or reviewing one or more models most appropriate for the user's 
needs. 

The starting point for using the consumer report is with the appropriate 
topology/evaluation matrix of the four in the report. If the user is not certain which 
matrix to use, the background sections for the four types of models should be read in 
order to pinpoint the appropriate matrix. 

An overview of the decision-making process to be used when using the consumer 
report is presented in Figure 1. This process is sequential as ordered from top to 
bottom of the figure. After the user has identified the appropriate matrix, the 
Types/Purpose, Processes, and Levels Focus characteristics are perused to identify 
models that contain the components necessary for the user's needs. Then the 
characteristics that have ratings are considered, and for those models that meet 
minimum criteria in relation to the user's needs, the profile data forms are reviewed. 
The final step is that of obtaining detailed descriptions of the one or more models 
that can be adopted/adapted for the user's purposes. This requires that readers 
contact the organization, association, or agency listed in the respective profiles. 
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Figure I 

Flowchart of the Decision-Making Process When Using the Consumer Report 



Information Source 



Decision 



Typology/Evaluation Matrix 



National 



Regional 



State 



LEA 



Type/Purpose 



Identify the approriate 
category and model, that is, 
select the appropriate matrix. 



Identify which of the three 
types/purpe ses to include — C 
(Complinace ), D (Diagnostic), 
and P (Performance) — or 
combinations thereof, and 
mark those models that at least 
contain the types/purposes 
identified. 



Processes 



Levels Focus 



Identify the processes and lev- 
els focus to be sought and mark 
those remaining models that 
meet these criteria. 





Ratings on Remaining 
Characteristics 












School Evaluation Model Profile Data Form 










Detailed Descriptions of 
the Models From Their Sources 





Ma 'hose models from the 
pre s step that meet mini- 
mu iteria on the 17 charac- 
ter!^, os rated, considering 
user's needs. 



Identify the models that meet 
the user's criteria, needs, and 
resources. 



Select the one or more models 
that can be adopted/ad;^ r ed 
for the user's purposes. 
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HI. NATIONAL MODELS 



Background 

School evaluation models listed as national n.odels originate from national 
associations and organizations, as well as from a federal government agency and a 
for-profit enterprise. For example, the National Association of Secondary School 
Principals attempts to improve the quality of the schools it represents through the 
development and dissemination of a school evaluation model that it believes will 
contribute to school improvement. Its School Climate Survey provides detailed 
diagnostic information at an individual school level. Other national associations, 
such as the National Association of Elementary School Principals and the American 
Association of School Administrators, have developed and disseminated guidelines for 
individual schools to follow in conducting a school evaluation, but no systematic 
national program has been implemented based on these efforts. Private enterprises, 
such as the National Study of School Evaluation, sell their various evaluation models 
to organizations (e.g., Southern Association of Colleges and Schools) and schools of 
all types, including private schools, to assist them in obtaining information about 
their schools for whatever purpose may be at hand. 

The National Catholic Educational Association (NCEA) does not accredit its member 
schools but does distribute a self-study instrument that was developed to assist all 
member schools in periodic self- assessment. Approximately 85 percent of all Catholic 
high schools are accredited by a regional accrediting association; therefore, the 
NCEA self-study tool is offered as a possible instrument for use in regional evaluation 
processes. 

The federal government, through its school recognition program, attempts to obtain 
data from schools in order to reward outstanding schools and to give them national 
recognition through an application of criteria established at the federal level. This 
national school recognition program was established in 1984, and annual awards are 
given to self-nominated elementary and secondary schools from every state. The 
criteria applied include results but do not require disaggregation by gender, race, or 
socioeconomic status. 

Trends 

Recent developments at the national level will undoubtedly have a bearing on school 
evaluation. This refers to the AMERICA 2000 program, which focuses to a great 
extent on individual schools. The most immediate impact of the program may well 
be a contraction of the outcomes of schooling to coincide with the six national 
educational goals. These goals emphasize traditional core subject areas (i.e., English, 
math, science, history, and geography) and seem to pay less attention to the goal that 
"every school in America will ensure that all students learn to use their minds well, 
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so they may be prepared for responsible citizenship, further learning, and productive 
employment in our modern economy." No specific mention is made of employability 
skills, values, art, or music. Further, there is little or no mention of the need to 
disaggregate data to ensure that all students will benefit from AMERICA 2000. The 
impact of this program on school-level evaluation, while uncertain at this time, is 
likely to be profound and may or may not be conducive to improved performance of 
all (or many) students on significant student performance outcomes. 

The typology/evaluation matrix for national models follows. (See Appendix A for 
profiles of models listed.) As a group the national models focus on diagnostic and 
compliance purposes. The equity (E) characteristic in accountability strands and the 
system focus tended to receive less favorable ratings. These models are weak on 
interactive linkages. Overall, these models received their best ratings on the 
characteristics of Planning/Decision Making. 
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IV. REGIONAL MODELS 

Background 

Historically, the major influence on school evaluation practices has come from 
regional accreditation associations, such as the North Central Association of Colleges 
and Schools. These associations have provided the majority of state departments of 
education, responsible for auditing schools in each state, with an organization that 
could be called upon to provide the expertise, organization, criteria, guidelines, 
personnel, and procedures for state agencies to assist districts in conducting self- 
studies and to gather the information to be provided visiting teams in order to be 
accredited. The contributions and impact of regional associations to school evaluation 
cannot be overlooked. Even though regional accreditation historically has been 
voluntary, it has been and continues to be a major force in school evaluation. 
Admittedly, all accrediting associations have developed compliance models that have 
helped the states to determine whether or not schools were meeting legal 
requirements established for their operations. Exceptions to this approach are very 
recent, but could prove decisive in schools' decisions to turn away from evaluation for 
simple compliance toward evaluation for diagnosis and performance. 



Trends 

The most significant development by regional accreditation associations in moving 
away from compliance models is the optional model now offered by the North Central 
Association of Colleges and Schools to the states in its region. This is a new model 
called the Outcomes Accreditation/Evaluation Model, which is outcomes based and 
which also provides for gathering diagnostic information helpful for school 
improvement. The adoption rate of this model has been remarkable and indicates a 
strong transition toward models that focus on both performance (outcomes) and 
diagnostic information and data. 

Another example of going beyond mere compliance review accreditation can be seen 
in the school evaluation model of the Southern Association of Colleges and Schools. 
It includes a relatively complete school improvement component that must be met to 
receive accreditation. 

It is anticipated that other regional accreditation associations will be moving in this 
direction in the near future. This trend is evident in the increasing number of pilot 
evaluations that focus on student performance now being conducted by these 
associations. Indications are that there will be widespread use of this approach as 
an alternative to compliance school evaluation models during the 1990s. This 
movement addresses most of the shortcomings of the traditional accreditation 
methodologies and appears to be promising in that it is tied to a formal school 
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improvement process and explicitly encourages disaggregation of data. However, the 
process is time consuming for school staff, and results may not sufficiently manifest 
themselves within the traditional five-year review cycle. 



The typology/evaluation matrix of these regional models follows. (See Appendix B for 
profiles of models listed.) As with the national models, the regional models tend not 
to have performance monitoring as a type/purpose, but focus on diagnostic and 
compliance. In fact, only one model reviewed, the North Central Association of 
Schools and Colleges-Outcomes, had performance monitoring as type/purpose. Only 
the Appalachia Educational Laboratory Model met the System Focus characteristic. 
Again, these models as a group are weak on interactive linkages. 

The models tended to be strongest on the metaevaluation characteristics. Only on 
the accuracy (A) standards did several models drop below "Meets" to a rating of 
"Partially Meets." Of the eight models reviewed, only two met the criteria for Product 
requirements in Planning/Decision Making, and only one received a rating of 1 on 
this characteristic. Regional models apply to all levels-E, M, and H-so they tend to 
have a comprehensive levels focus. 
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V. STATE EDUCATION AGENCY MODELS 

Background 

In contrast to regional, voluntary accreditation of schools, state evaluation of school 
systems and schools is part of an effort to guarantee that the monies expended by the 
various states on education are applied to the purposes for which they are intended. 
The result has been a move toward a mandated rather than a voluntary process on 
a regular schedule and an evaluation system based on monitoring the standards and 
rules governing the use of state funds. Since education is a state responsibility and 
because virtually every state expends the bulk of its budget on education, states have 
a legitimate concern with the quality and quantity of the overall education effort. 
With mounting indicators that many schools were not adequately serving their 
students and that the level of student performance was on a decline in some school 
districts, state legislators began to insist on a performance check of local education 
agencies. 

In the 1330s, the national mood on schools and school evaluation has shifted. The 
accountability movement is in full swing, with increasing demands for school districts 
and individual schools to improve their products. The six national goals adopted by 
the nation's governors and the U.S. Department of Education are providing the 
backdrop for the development of a national plan for education, with states and local 
districts urged to develop specific goals, objectives, and accompanying strategies in 
support of the national goals. This, in turn, has influenced the development of new 
school evaluation models that are being examined and tested around the country. 
For example, the extant Georgia Department of Education Comprehensive Evaluation 
System (CES) was evaluated with the knowledge that the Quality Assessment 
Component of the system was still in the developmental stages. Project staff could 
not wait to receive it for review because its completion has been indefinitely 
postponed due to budgetary constraints. Also, this report does not include a review 
of Kentucky's compliance model, because it was recently declared inoperative by the 
state legislature. In contrast, the University of Michigan model for accrediting 
secondary schools in Michigan (which has existed in one form or another since 1871) 
has been retained even though the state department of education, which approves 
elementary and middle schools, is supposed to assume responsibility for evaluating 
secondary schools sometime in 1992. Michigan has yet to develop a system for 
evaluating its secondary schools. 

The majority of state department of education models continue to focus on 
compliance. An underlying assumption is that if the compliance standards are met, 
a quality education can be provided, so most states have developed a demanding set 
of compliance standards for local school systems. These deal with such things as the 
number of volumes per student in the school library, number of stations in science 
laboratories, certification of the teachers assigned to specific subject areas, allocations 
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for supplies and equipment provided to individual schools and the district, courses 
offered in the school, number and type of Carnegie units required in specific curricula 
(e.g., college preparatory), and other input measures that have been a part of the 
tradition of American school evaluation. 

As educators began to question the veracity and quality of the existing evaluation 
processes and models, efforts to focus at least a part of the evaluation on student 
outcomes began to emerge. However, most evaluations are still conducted by utilizing 
the compliance model as the focal point. Part of the reason for the predominance of 
this approach is the traditionally held concept that quality is best measured by inputs 
and that quantitative measures can be translated into qualitative summaries. Thus, 
for decades the most common and easiest way for states to measure the effectiveness 
of the educational system has been to analyze the quantity of resources provided and 
the quantitative evidence that indicates whether or not predetermined standards 
were met, and thus to make inferences about the quality of a school. 



Trends 

The educational reform movements of the 1970s and 1980s, which were largely state 
level reform movements mandated by the various state legislative bodies and/or by 
governors with an interest in education, began to focus on qualitative evaluation 
using student outcomes. Unfortunately, these incipient efforts placed an inordinate 
amount of trust on the definition of quality as an interpretation of results from state- 
mandated tests given at various grade levels. As a result, little was done to establish 
broader definitions of quality to enable a comprehensive and complete evaluation of 
the educational system and an individual campus. California, with its grouping of 
like campuses for purposes of comparison, and Texas, with its effort to relate 
qualitative measures to the composition of the student population of a particular 
school, made early efforts at establishing reliable qualitative indicators. 

Such efforts were the start of an emerging trend that continues to this day. As a 
result, the discrepancy between what the reform movement requires as evidence of 
excellence and what the data of compliance models have traditionally provided is 
becoming clearer, thus creating a need for an increased effort to develop qualitative 
indicators as measures for evaluating and improving schools. 

As the work of the effective schools movement focusing on school improvement gained 
support across the country, the need to plan for improvement on a school-by-school 
basis became an added factor in developing qualitative indicators for evaluation and 
school improvement. State after state adopted the effective schools correlates as the 
measuring rods for school evaluation, and a generation of educators began to use 
th se correlates as the basis for school improvement. A& a result, new system-wide 
£ individual school evaluation models have been developed by some state 
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departments of education after being required to undergo dramatic state-mandated 
reforms (e.g., Texas, Indiana, Connecticut, California, New York, Tennessee, and 
others). It must be noted, however, that some states (e.g., Colorado, Nebraska) allow 
schools to select from more than one model to meet state requirements. That is, even 
though some states have developed their own school evaluation models, they permit 
schools to use models developed by such organizations as the North Central 
Association of Schools and Colleges, the National Study of School Evaluation, and the 
Self-Study Guide for Catholic High Schools in lieu of the state model. 

Typically, state evaluation models attempt to meet two basic needs: (1) the ongoing 
compliance tasks for which the states continue to have responsibility and (2) meeting 
the emerging qualitative evaluation demands for determining the authentic effects 
of a school system's and/or school's educational program on students. These state 
evaluations in which the traditional compliance measures dominated the thrust of the 
evaluation have been expanded to include a diagnostic analysis as well as a 
determination of productivity as assessed by standardized test scores, graduation 
rates, etc. The Connecticut Model, for example, which suggests a varied evaluation 
approach depending on the clientele of a particular school, is predicated on school 
effectiveness definitions and is both formative and summative in its intent. 

While it is inappropriate to generalize about all state evaluation models, it is 
apparent that the majority are compliance models and will remain so for some time. 
However, it is also clear that, under mandates from reform legislation, models are 
being developed in the states that take into account the diagnostic needs and 
performance expectations of school systems and individual schools. Much work 
remains to validate and refine this relatively new endeavor. Unfortunately, too many 
professional educators in too many states find themselves on the outside looking in 
as new school evaluation models and procedures are proposed and written into law 
by legislative committees far removed from the realities of schooling. The enactment 
of the Kentucky Education Reform Act of 1990 is an example. Its accompanying 
administrative regulations made the extant state compliance school evaluation model 
inoperative. In attempting to ensure school accountability through a new 
performance-based assessment program (to be operational no later than the 1995-96 
school year), a controversial policy change was legislated. It removed authority of 
local school boards to make personnel decisions relating to dismissals and placed it 
in the hands of a "distinguished educator' 1 assigned to a building and/or central 
administrative offices. The distinguished educator has unilateral power to remove 
teachers and/or administrators including the superintendent. Such dramatic policy 
changes tend to disrupt or even make inoperative existing evaluation models without 
providing or even implying a feasible alternative. 

The typology/evaluation matrix for state models follows. (See Appendix C for profiles 
of models listed.) The state models show considerable variability in ratings across 



26 



42 



models and somewhat less variability within models. Thus, the stronger models tend 
to be consistently strong and the weaker models consistently so across characteristics. 

As a group, state models are weak on interactive linkages. Of 22 models reviewed 
(Michigan had two models listed together), only 8 met the criteria for interactive 
linkages. Most models are satisfactory on their data focus with quantitative stronger 
overall than qualitative. The intended focus for most models was also found to be 
satisfactory. About the only other discernable pattern in ratings across the models 
is in the accountability strand of equity (E). Models tend to either more than meet 
or just partially meet this characteristic. All models addressed this characteristic; 
there were no 4 or 5 ratings. So, this characteristic is either only partially addressed, 
or more information is obtained and procedures are more than adequate to meet this 
criterion. 



27 



c 

'33 
> 

d 

c 

o 

oj 

_d 

> 

CD 

CD 



o 



cd 
cj 

1-1 

c2 

CD 



CO 

G 
o 
a 

CO 
CD 

u 

CD 

CO 

o 



G 



cd 

CO 

cd 
cr 

CD 
cd 

<2 

CD 

-a 

cd 

S 



e 

G 
o 

• <— < 
CO 

'3 

CD 
>> 

5-. 
O 

03 

a 



cd 
a 



CO 
CO 

CD 

CJ 

o 

u 

a 

c 
o 



cd 
> 

CD 

fa 



05 

c 
2 

CO 

CO 

O 

u 

a 

CD 

. 



«m CD 
^ CO 



CD 

e 

a 

o 



CO 

d 

CO 



CO 

CD 



.2 -2 



2 ° 



o 

CO 

G 



CD - 

a > 
G © 
cd ^3 

cd 



_ 

S * S 



CO 

c 

CD 

6 

• 

u 

CD 

a 
x 

CD 



o 

O 
O 



G cd 



U CD 

cd 



*-» 4^ — cd 
JJ G ^ a) 

2-c 5 



•e a 



33 



CJ 



cd 



a 

cd 
3 

"cd 
> 
CD 

CD 

u 

cd 

CAfl 
G 

• ^ 

cd 

S 
g 

o 

a cd 



9 



CD ^ 



a 
d 

a 
s 

o 
o 

CO 



CD 

cd 
a 



bp 
CD 



O 

o 

T3 CO 

<D , 

3? 3 

q 5 
o cd 



e ^ S 9 « <^ S cd 

CD.SP^ 'Q'C > ^ ^ 



CO 

— '3 



cd 
cj 

a 



cd cd .2 



cd 

G 

CD 
cj 

CO 



5 co 



CO cd cd 



CO 

cd 



o 



CO 



a 

o 

CO ^ 

G 



G 

CD 



CO CO 



CD 
T3 



CD 

u 
d 

o ^ 



G 
G 



CO 
CO 

*o cd u 

2 => * 8 

ft CO ^ X 

3 O 



o cd o S *j ^3 co .2 



s ,CD d ft G 



<D w 
U CO CO 



jG 

CD 

cd 



CD Z> 

CO 



^ ^ ^ r| 

G G G 

CD CD 



cd a* 



CO CO 
CD CD 



CO _ 

G ^ 
O G 



u d 
^ ft G 

ft^ 
d 

co 



CD 



CO CD 

re 
si a| 

^ cd "5 -rj 

cj .w 4-» • 



o * 2 -5 53 d 



0) 2 g s G 



Cd 

S 'C 
cd 



G 

o 

CO 



CD CD 



cd 

CO 

u 
o 

G 
cd 



1.1 

cd 



co d - 

co-^3 



"T7 

CO CO CD 

cd cd g 

CJ O CD 

^-N G ^ 
w CD CD - 

cd > Xi XI 

^ d d 

<2 cd cd cd 

o -< CO CO 

C ctf fcfi 3 ^ 
CD o CO Cd g CD 



cd 

ft 

5 * b 
d cj cd 

o o 



ft cd ^ 



o co 
d 

CO 



a 
d 

CO 



A -2 ^ ^ 
^ ^G >, cd ^ 

4-> 



O CD H 

2 > 55 

cd *"Q cj G 

■8 3 | 2 gi 
o . M o g I -3 

o 

-O cd 



G 
o 



. 2 cd 



CO 

ft 



T3 

CD 

s 

CD 
U 



O 
O 

o 

CO 

u 

CD 

cd 

G 
» *w 
u 
o 



CD 
US 

cii 5 



G 

CD 

e 

CD 
> 
p 

a 
S 



o 
o 

"3 

CO 

T3 
G 
o 

>> 

CD 
-Q 

cd 

CD 



S T3 
G « 



co 



txp 

CD 
CO 

s 

OS 

& 
o 

5-4 

ft 

U 
O 
ft 
ft 

d 

CO 



u 
G 

CD 

u 

a 
'C 
d 

CJ 

cd 



CO 
CD 

u 

co ^2 

CD ^ 
CO 
CO 

cd 



CD 

CJ 

o 
u 

ft 

ft 
d 



CD 



d 



cd S fD cd 2 

<D 3 CD & ° Cd > 

* - * i \i z 

cd 



CO 



CO 



T3 .S .2 



cnCd^GddG.^G'G 
8 •^•8 I a T3 ||| 

ft cd 



o 

CO cj - 

co ^ t3 



co 



§ 3 3 3 

^ S <n q.S o oiJ 2-3^.2 M <2 ctio 



CD d 

§ 3 



G ^ O 

^ cd CD 
G ^ 
cd 3 

CJ 



CO 

d 

o 



J s 

cd 

> o 

<U CD 

co TJ 

"co S 

CO ^ 

CD co 

•2 d 



3 .2 ^ 



'5 ^ 

a a 

o <2 
qi -a 

2 % 

CO cd CD 

.2 h J 

4-1 CD 

'S -g -a 

3 « S 

S ** T3 
0.13 0) 

o aj S 
co 4j S3 

s § s 

® U 

(3^2 £ 
cd cd .2 



!> 



> 



CD 

g 

05 0J 

. o 

CD CUD 

co G 

^ o 

.2 cd 



.52 

G CD 

.'5 ^ 

^ CD 3 



O 
cj 

CD 



CD 

cd 

u 

cd 

o 



cr 

CD 
u 

CD 

cd 



CD 



O 

o 



& K> 
Got 

O - 

fa CD 
ft cj 

cd 
u 



bJO 
G 



co 

CD ^ 

co G 

co cd 
cd cm) 



CD CO 

> "~ 
o 



G 

o 

cd 



8 s 3 g 

2* S «S >» 

* j§ is « 
S « c c 

* « 3 53 

<302 u 

g -a 

** 2 n-i 

2 « -3 



09 




eu <u <u 
-C -C -C 

5-» 6-* E-» 



So &3 Q 



Tj 1 



VI. LOCAL EDUCATION AGENCY MODELS (DISTRICTWIDE, CAMPUS) 



Background 

The development of models to evaluate individual schools (i.e., campuses) received its 
major impetus from the research conducted by proponents of the effective schools 
movement and the school improvement process developed to implement the research 
findings and the inferred effective schools correlates. Even though this research was 
begun in the 1970s, school improvement efforts and their corresponding evaluations 
were primarily a phenomenon of the 1980s, with much of this work continuing to the 
present. 

The major, overriding goal of school improvement efforts across the United States has 
been to achieve quality and equity in teaching and learning for all students within 
a school and a school system as a whole. This requires all schools within a district 
to establish school improvement teams-composed of faculty, administrators, and, 
sometimes, parents-responsible for developing and implementing a school 
improvement plan. Unfortunately, it is not a common practice to hold these teams 
responsible for evaluating their effectiveness. In fact, it is a rather recent 
development that districts themselves have given serious attention to developing and 
implementing school evaluation models. 

School evaluation models (sometimes referred to as an audit or accreditation process) 
developed and used by local education (LEA) agencies are gaining in popularity for 
several reasons: (1) the need to determine the degree to which a school improvement 
plan is being implemented, (2) the need to examine the degree to which quality and 
equity have been established or increased within a school, (3) the need to obtain 
evidence of school effectiveness in terms of student outcomes, (4) the need to 
determine the presence or absence of the effective schools correlates, and (5) the need 
to gain insights for school improvement that result in recommendations for possible 
adjustments in implementation of standards. In addition, some districts conduct local 
school evaluations (audits) in preparation for a required state accreditation visit. 

A significant and powerful influence that is creating a press for the development and 
implementation of school evaluation models is the growing movement toward school 
site-based management throughout the United States. Because of the autonomy 
granted individual schools under this type of restructuring within a district, there is 
an even greater need to know how school-based management contributes to a school 
improvement plan. 

School evaluation models tied to school improvement, then, are in almost all 
instances diagnostic in intent and have a formative perspective in their application. 
In addition, they tend to be evaluations of process and product, with a greater 
emphasis on the former. That is not to say, however, that summative evaluation 
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decisions are not a goal; i.e., decisions to continue or not continue certain processes 
in teaching and learning. In spite of these summative intents, however, decisions are 
often guided by important outcomes with no clear indicators of school improvement 
or by a focus on a restricted set of measurable outcomes. 

Because of the nature of school evaluation and its intent, several issues arise in 
relation to it. These relate to such things as who should be responsible for developing 
the school evaluation model and for carrying out the evaluation itself. The concerns 
here have to do with whether or not local staff (administrators and teachers) have the 
expertise and the time necessary to develop and implement an evaluation process 
that meets established standards for such evaluations. The issues of inherent bias 
in data collection and the desire to present positive results cannot be ignored and has 
led to the employment of districtwide teams to review, analyze, and verify the data 
presented in reports of school improvement teams several weeks before visiting a 
school. 

Trends 

However school evaluation is guided or conducted, the relatively recent and mounting 
pressures for schools to identify indicators of quality in teaching and learning, and 
for broadening assessments of both professional staff and students to encompass the 
emerging characteristics of "authentic assessment" will force school evaluation 
practices to change significantly regardless of their level of origin and 
implementation. School evaluation models developed in support of the effective 
schools movement and its corresponding school improvement efforts still rely 
primarily on quantitative data, in spite of considerable reference to quality in the 
literature. A balance will have to be struck between quantitative and qualitative 
data in school evaluation models to provide more in-depth meaning to results and 
thus attempt to meet the demands of American society. 

Evidence of quality in our schools will undoubtedly have to be j~ vided, to a great 
extent, by teachers who can document daily and periodic learner behaviors that 
would not be recorded through limited observation by external evaluators. The 
education and training of teachers to develop indicators and implement procedures 
for assessing all facets of learner behavior is critical to the successful development 
and implementation of local school evaluation models. 

The typology/evaluation matrix for local education agency (LEA) models follows. (See 
Appendix D for profiles of models listed.) As a group, LEA models focus heavily on 
diagnostic and performance types/purposes. They tend to be comprehensive, 
addressing all three levels' foci, and for the most part the models are distinctive for 
E, M, and H. There is considerable variation across models on interactive linkages. 
Out of the 15 models, 10 at least "meet" the criterion; and, of these, there are three 
that "more than meet." 
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The consistency of ratings within models tends to be greater than consistency across 
models. That is, a model strong on some characteristics tends to be strong 
throughout and vice versa. Within Planning/Decision Making, models tend to be 
strong on context and somewhat weaker on the other three characteristics in this 
category. LEAs had the highest proportion of models of the four groups, exhibiting 
unique strengths. A possible reason for this result is the more localized focus of the 
models, rather than addressing a broad spectrum of schools such as the national 
models. 
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VTL OVERALL FINDINGS/OBSERVATIONS 



In the immediately preceding sections, characteristics of models were discussed by 
categories as presented in the four matrices. However, there are other ways to 
analyze models and one such is Richards' (1988) Types/Purpose classifications. 
Again, according to Richards there are three main types of school evaluation models: 
those that (1) monitor for regulatory compliance, (2) monitor for instructional 
diagnoses and remediation, and (3) monitor for school and student performance. The 
findings presented below follow these types/purposes. 

Compliance Monitoring Models 

Compliance monitoring will undoubtedly continue to be required to ensure that legal 
requirements established by the states are being met and to provide information 
about the implementation of special programs serving youth with special needs. The 
models developed by national organizations, regional accrediting associations, and 
most state departments of education are primarily compliance models. They are 
similar in design and approach, requiring a self-study document to be submitted by 
school districts to an external agency, followed by an on-site visit by a team of 
evaluators. These are basically "input" models that focus on quantitative data 
relating to the number of certified teachers, classified staff, support personnel, their 
education and training, and their relationship to their professional assignments. In 
addition, such models require data on financial resources, the condition of the 
physical plant, and the quality and quantity of instructional and library resources. 

Compliance models are designed to comprehensively and accurately meet their 
intended purposes. Nevertheless, in light of new demands for school improvement 
and accountability, models emphasizing compliance that require only input data are 
of little use in meeting the needs of America's schools. Demographic changes in 
student populations have resulted in more student diversity, and a weakness of 
compliance monitoring models is that input data alone are too limited to assist 
schools in serving better those students with unique needs-needs that impact the 
teaching-learning process. 

It can be argued that compliance monitoring is the easiest, least intrusive, and most 
inexpensive approach to evaluating schools; and these features may provide some 
attractiveness for the models. But, generally, compliance monitoring does not 
contribute to meeting the developmental needs of students or the improvement of 
America's schools, nor does it provide data on outcomes needed to evaluate the total 
performance of a school and/or a school system. Compliance models, therefore, 
primarily serve to audit inputs in relation to school resource availability. 

Of all the models reviewed, only the Wisconsin model in the State Models Matrix had 
compliance monitoring as a single Type/Purpose, So, generally, models are not 
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limited to compliance monitc tig, and the other Types/Purposes (D, P, or both) make 
the models more appropriat >r evaluating today's schools. In sumrr iry, compliance 
monitoring by itself serves a very limited purpose when interpreted *n the context of 
school and societal needs to be met by school evaluation. In this regard compliance 
monitoring has a serious limitation. 

Diagnostic Monitoring Models 

The " effective schools" paradigm is perhaps the most widely used for school 
improvement efforts in the United States. Practically all of the local education 
agency (LEA/campus) school evaluation models reviewed focused on the correlates 
identified with school effectiveness. The diagnostic monitoring models are designed 
to look at the process being used within a school to achieve established goals and 
objectives relating to student performance and to contribute information to be used 
in school improvement plans. This applies whether norm-referenced test results or 
objectives-referenced test results are used as information sources for student 
performance. An important distinguishing feature of these models is the 
disaggregation of data by gender, race, SES, etc. Several of the state models also 
require that school systems and campuses prepare improvement plans based on 
school profiles engendered by the management information systems now in place in 
some states (e.g., Texas). Diagnostic models call for a context analysis of where a 
school or a district considers itself to be in relation to achieving goals and objectives 
adopted by the governing body. Schools and school districts can and often do bring 
.in outside consultants to assist in a diagnostic evaluation. More often than not this 
type of evaluation is self-initiated and not imposed externally. However, there are 
a number of state and local models (e.g., California; Texas; Indiana; North Carolina; 
Spring Branch, TX; Omaha; Ann Arbor, MI; Des Moines; Prince George's County, 
MD) that require their schools to develop school improvement plans based on the 
effective schools paradigm as a basis for evaluation. Nevertheless, the emphasis is 
on self-study with a focus on consensus goals and objectives that have been adopted 
and approved at both the school and district levels. 

The strengths of diagnostic models are the commitment of professional staff to school 
improvement and the shared responsibility it generates among them. In addition, 
these models are practically required to go beyond the diagnostic stage to look at 
student outcomes in order to determine whether or not improvement is, in fact, 
taking place. Diagnostic models, therefore, are primarily formative in nature and 
attempt to formulate relationships between and among context, input, process, and 
product for school improvement. 

The shortcomings of these models revolve around the amount of staff time required 
to implement them properly, and the difficulty of "requiring 1 all staff to agree 
adopt whatever instructional approach or working condition that is being considers 
Also, the information obtained through these processes and the improvement pla 
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derived therefrom may not be of significant value to school improvement due to a lack 
of fiscal and human resources-a condition that leads to frustration on the part of 
professional staff who now have unmet expectations. Schools involved in site-based 
management are invariably tied to diagnostic monitoring when evaluating schools. 
In practice the evaluations meet with mixed results. 

Performance Monitoring Models 

Over the past decade, the growing pressures for accountability from state legislative 
bodies and from state boards of education have resulted in the use of statewide, 
standardized achievement testing across the United States. The use of such tests has 
greatly influenced school evaluation practices, with more and more states moving 
toward performance-based models for evaluating schools while still conducting their 
legislatively mandated compliance monitoring fimctions. Recently developed state 
accreditation models, such as those in Indiana and Texas, which encompass 
compliance monitoring as well as monitoring student performance outcomes, also 
evaluate how well a school system or a school meets its established objectives as 
measures of success and/or failure. Performance models, then, are considered 
summative in nature but do provide the bases for school improvement in a formative 
sense, in that they establish new levels of expectations for the subsequent evaluation 
cycle, but they do not prescribe or suggest how to attain these expectations. 

The accountability movement has also influenced regional accreditation agencies such 
as the North Central Association of Colleges and Schools, which now has two 
accreditation models-a compliance model and a model that also has performance 
(outcomes) monitoring. Both models include diagnostic monitoring. Clients have the 
option of selecting either model. Of special interest in the more recently developed 
performance models, such as the Texas Education Agency model, the Indiana State 
Department of Education model, the Spring Branch Independent School District 
model, and the Des Moines City Schools model, is the push for obtaining 
disaggregated data so that evaluators can determine the effect of school programs on 
the various constituencies within schools. It is also interesting to note that, as school 
evaluation focuses more and more on outcome measures, there is correspondingly less 
emphasis on compliance issues. 

There is little question about the important effect that the accountability movement 
has had on school evaluation in America. It has raised a legitimate concern about 
the quality of the product of our nation's educational effort and has forced evaluators 
to look at important dimensions of schooling that were previously ignored, and to 
develop new insights, approaches, and tools to be used in evaluating schools. This 
has been a very important contribution to the field of evaluation, but much remains 
to be done to ensure that any shifts in evaluation emphasis (for example, from 
compliance monitoring to performance monitoring) or any new approaches that may 
be developed are truly positive forces for educational improvement. The education 
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profession is well advised to ensure that key indicators of quality are incorporated 
into school evaluation models. 

Common Weaknesses in School Evaluation Practices 

A review of the models included in the typology/evaluation matrices of this consumer 
report revealed some common or general weaknesses and voids in the models. There 
was no doubt about which models were stronger overall than others, but the majority 
of the models, regardless of their intended purposes, suffered from serious deficiencies 
that need to be addressed if school evaluation in America is to constructively serve 
the education professionals and youth in our schools. School evaluation will not be 
reformed overnight. Improved models will be developed through successive 
approximations toward ideal models, and improvements will come by building on 
present models, capitalizing on recognized strengths, and eliminating weaknesses. 

While the accountability movement has focused attention on the need to develop 
outcomes-based models of district/school evaluation, the actual development of such 
models is often skewed by developer bias and perception (e.g., of local outcome 
statements). Also, inconsistent definitions of what is being evaluated and the 
meaning of various terms contribute to a general confusion about evaluation. In 
addition, while new state level standards for local schools are being established at an 
increasing rate across the country, there is h,Ae evidence of ownership at the local 
level or of a systematic and appropriate response to such directives. As a result, the 
models now in use generally reflect the impatience of the developers and their 
penchant for quick fixes rather than systematic improvement. 

Self-evaluation efforts, while important and needed, do little to assist the staff in 
improving its evaluation skills and understanding. This is due to a lack of staff 
development that includes education and training in school evaluation skills and 
processes. 

The common weaknesses and/or voids common to school evaluation models include 
the following: 

o No clear indication in the majority of the models that evaluators had received any 
significant training in school evaluation 

o Little evidence that evaluations are based on any acceptable and recognized set of 
standards for evaluating programs or products 

o Limited evidence that a systematic approach to school evaluation is understood or 
practiced 



36 



58 



o Limited attempts to address issues of quality in any meaningful way, including a 
wide array of considerations not adequately addressed: 

- The content taught in a field/area of study encompasses the most recent 
advances in the field. 

- The objectives being addressed in a field/area of study are of significant 
importance, appropriate, and generalizable. 

- The instructional faculty are performing competently in taeir areas of 
responsibility, and their evaluations are linked to student performance. 

- Staff development provides the education, training, and follow-up to 
contribute to excellence in a school. 

- Needs are assessed :n a manner that identifies the developmental needs 
of students above and beyond gaps in achievement levels (e.g., uses 
qualitative as well as quantitative measures). 

- An assessment uses disaggregated data for ali students in all areas of 
study regardless of race, ethnicity, gender, or disability. 

• - Plans for school improvement are assessed to determine if they are 
realistic, feasible, and will contribute to overall excellence. 

- The data collected include information/data related to authentic 
evaluations of students and personnel. 

o No indication that a metaevaluation of school personnel evaluation practices was 
conducted or even suggested. 

The above discussion of weaknesses is not intended to imply that evaluation model 
developers have not worked hard at their task. School evaluation is a complex and 
comprehensive enterprise. It is impacted not only by educational forces but by 
political forces within and without the schools. Listing weaknesses and then 
attempting to eliminate them can be an effective part of improvement. It is with that 
goal in mind that the above observations were made. Unique strengths of selected 
models were indicated in the matrices. By capitalizing on those strengths and 
eliminating, or at least countering weaknesses, efforts to improve school evaluation 
models can move forward. 
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VTIL RECOMMENDATIONS FOR IMPROVING MODELS 
FOR SCHOOL EVALUATION 



Schools today are a direct outgrowth of an evolutionary process in teaching and 
learning that has been based to a large extent on tradition, vested interest, and some 
mythology about what is best for America's youth. Along the way intermittent 
attempts to "reform" our schools have emerged and are generally regarded as failures, 
even though education professionals have placed great stock in what they have done. 

At present our country is overrun by attempts to dramatically improve schooling. 
The current onslaught was spearheaded by the so-called accountability movement 
that began a press for higher achievement. Students in grades four, eight, and ten 
are tested in basic skills areas, with some variations that generally include math, 
science, reading, and English. This approach to school improvement has been 
sustained and augmented by calls for "restructuring" America's schools in order to 
break the mold of traditionalism, which is considered the major stumbling block to 
school improvement. 

So far, the major weapons in concerted school improvement efforts have been the 
widespread use of achievement tests, such as the those developed by the National 
Assessment of Education of Progress (NAEP), and those developed by individual 
states, such as the Michigan Assessment of Educational Progress (MEAP), Also, 
school evaluation conducted by state education agencies is now assuming a major role 
in school improvement efforts in many states; and, as stated earlier, a growing 
number of states are now under legislative mandates to reform their school 
evaluation practices to provide more consistent, discrete, and useful information that 
can contribute directly to school reform and improvement. 

This said, it behooves those concerned with school evaluation to be aware of what has 
been learned with the hope that developers of new and emerging school evaluation 
models will pay heed to suggestions for improving current evaluation practices. The 
discussion below is an attempt to contribute to the improvement of school evaluation 
with the realization that the recommendations may not apply to all school contexts. 
Adoption is left to those in the field. 

Education and Training 

States departments of education can and should participate in educating and training 
school staff to become better evaluators. When a state model is defined, and 
especially if its implementation is legislated, a state department of education can 
provide the necessary education and training for the local staff to enable them to 
participate constructively in the evaluation process. It may be argued that there is 
an obligation on the part of the state department of education to provide the 
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necessary training. Texas, for example, has developed a model called the Texas 
School Improvement Initiative (TSII) in which local administrators and teachers are 
assembled for a week-long seminar for education and training in the application of 
the state accreditation model. TSII also provides the basis for the recognition of 
exemplary schools across the state. This process is given credit for being the most 
effective tool utilized in encouraging change and school improvement in Texas over 
the past decade. 

Not all evaluation models originate at the state level; but regardless of where a model 
originates, it is necessary to provide the training for implementation of the model. 
Developers of the model should make provisions for the necessary training, whether 
provided by the developers or by an equally competent source. School evaluation 
often is not one of the most popular issues among educators. A contributing reason 
for this is that those designated for the task often are not adequately trained nor are 
they given adequate time and resources. Training is essential and school evaluation 
should not be undertaken without adequate training, time, and resources. 

School/University Partnerships 

Cooperative relationships between local school districts/schools and colleges and 
unive rsities must be developed and improved. Professors with expertise in evaluation 
must collaborate with school professionals in the development of evaluation models 
that are effective, efficient, and relevant in meeting the demands for school reform 
and improved quality. Research relating to authentic indicators of quality must be 
conducted and directed toward the most pressing problems facing the schools in the 
United States. Recognition that the school is the most compelling laboratory 
available to the researcher and that the best research is that that can be directly 
applied to meet a school's needs must occur. The classroom teacher and local 
administrator do not have the time, energy, nor expertise to conduct the kinds of 
research needed to provide the basic information to reform the educational system. 

Cooperative governance mechanisms (e.g., principals and teachers) are now being 
explored and adopted across the country. This often calls for participation in 
evaluation of school effectiveness by teachers, which means that the professional 
needs of practitioner candidates have suddenly changed, and the universities and 
colleges must respond to this new reality by including meaningful instruction in 
evaluation for their teacher and administrative candidates. 

Evaluation and Restructuring 

The form of whatever new school structure is adopted should depend on evaluation 
findings. Of critical importance, however, is that evaluation of local schools and 
districts should be based on an individual school's improvement and educational 
plans and desired outcomes so that it is held accountable for those things it indicated 
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it would do rather than sc le extraneous standards over which the sc - ol/district has 
no control. Local plans must be predicated on a context analysis tk ; generates the 
information necessary (e.g., needs assessment) to develop the local evaluation plan. 

Recommendations for School Evaluation 

The following recommendations apply to the development of any school evaluation 
model. They are an outgrowth of the review of the models submitted for this 
consumer report. 

o While compliance needs can and should be minimized, they will continue to be 
required as part of the school evaluation process. Regardless, more time and effort 
should be given to value judgments relating to such issues as quality, equity, and 
fairness, while reducing data gathering on such things as volumes in libraries, etc. 

o The analysis of a school's student population needs will have to go beyond 
compliance standards and regulations established by federal and state laws and 
district policies encompassing these mandates. Although compliance requirements 
will be part of a school site improvement plan, variations or letters of agreement 
may be necessary to refocus school evaluation models in order to identify trends in 
student outcomes or performance and to assess a school's success in meeting 
students' special needs. Therefore, a mechanism must be in place to challenge 
compliance requirements that present hurdles to getting at issues related to 
student performance and needs. 

o A total systems analysis should be considered in the development of an appropriate 
school evaluation model for a district or its schools. Databases, established through 
a systems analysis and designed for each school site, should be the focal point of a 
school or district's school evaluation model. These databases should reflect 
demographic information, disaggregated test score results, attendance data, school 
dropout information, assessed needs, and results of school climate surveys. 

o Evaluation models should contain a school improvement component that focuses the 
evaluation toward outcomes rather than input measures. This component should 
relate to all domains (i.e., affective, cognitive, psychomotor). Diagnostic evaluations 
or self-studies by school-based councils (e.g., school site stakeholders) should focus 
on a systematic view of the effect educational programs have on student academic 
and social growth. This will mean moving toward an "authentic assessment" of 
students that relates to a comprehensive set of performance measures and 
previously assessed student needs. 

o School evaluation models should encourage wide participation from those 
individuals and groups who will be affected by the application of such models. 
Participation couid include principals, teachers, support staff, parents, students, 
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community agency and business representatives. Such involvement will provide 
different but important perspectives to diagnosing student and school site 
information, establishing priorities, creating action plans, and will encourage others 
to assume ownership in school evaluation and improvement. 

o Outcome measures should address more than achievement test scores and should 
contain measures of quality that relate to a broad range of student and professional 
staff performance and growth. This should include the assessment of resources 
allocated to programs and the results of such programs serving students in greatest 
need to ensure educational equity. 

o School evaluation should encompass an internal perspective and self-assessment, 
with steps taken to guarantee that all key players are adequately trained to carry 
out their assigned roles. A self-stu dy should precede a formal outside visit by at 
least one year to allow for any corrective actions to be taken prior to an external 
evaluation. 

o A school evaluation model must allow for periodic examination by a properly 
trained outside review or audit team. This team would be charged with 
determining whether a school's improvement plan is feasible and doing what it 
purports to do. 

o All evaluation models should contain a strong context evaluation component in 
order to establish the conditions within which to assess school effectiveness. 
Models must take into account the fact that educational decision-making occurs 
within the framewcrk of a school system and its surrounding community. Problems 
and opportunities facing each school district and its school sites must be identified 
and evaluated systematically with the acknowledgement of organizational 
relationships and decision-making consequences. 

o There must be a clear understanding of a district's relationships with organizations 
outside the district (e.g., state department of education) during the model 
development process. 

o The opinions and views of all key stakeholders in a school's evaluation should be 
assessed. This includes students, parents, the professional staff (teachers, support 
staff, administrators), the school board, and the community. 

o A regular school evaluation cycle should be established, with a comprehensive 
evaluation taking place every five or six years. Paper audits during the intervening 
years would call attention to special developments and/or problems that need 
attention. 

o Realistic finances for an evaluation must be included in ths school budget. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source School Climate SiigffiylSliidanfc, Tparhgr f Parent 

Sfltigffl^inn Snrvpyg 

Origin (Sponsor) National Association of Sp^ondary Srhool Principal^ 

1904 Association Drive . 

Ragtrm, VA 22021 

Level Qzl2 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

'The model of the school environment developed by the task force goes beyond a 
simple consideration of school climate to encompass a full range of inputs and outputs 
to the process of school improvement." 

"Successful leadership" begins "at the building level." 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Evaluation must be an ongoing, building level process that looks at the climate and 
other variables in and around a school. The climate of a particular school does affect 
the effectiveness and efficiency of that school. 



EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Context, input, output, process 
Qualitative - school improvement 
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PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

Mainly formative for self- improvement. 
ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Rigid: Utilizes surveys and measures specific criteria. 

Flexible: The individual school can io all its own sccring and interpretation 
of the data right in the school. 

SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

NASSP 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

What makes an effective school? What are stakeholder perceptions of a school's 
culture? 

EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

Subscales: Teachsr-student relationships, security and maintenance, 
administration, student academic orientation, student behavioral values, guidance, 
student- ~ *er relationships, parent and community-school relationships, 
instruct :al management, and student activities. 
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MAIN EVALUATION METHODS USED IN THE MODEL 



What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveyf 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Principals, students, teachers, and parents fill out surveys. 



MANUALS. FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

1. Examiner's Manual (1987). 

2. Sampler Kit: Comprehensive Assessment of School Environments . (Included are 
the scoring sheets and surveys.) 

3. Technical Manual. (1989). 



MOST DISTINCTIVE FEATURE(S) 

" What main points characterize this model? 

This model is an interactive, building level assessment, self-improvement model, 
designed for use by individual principals and schools. 



SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 
Origin (Sponsor) 



Level(s) 
APPROACH 



SelfeStiidy Guide for Catholic High schools 

13S1; third printing, 1Qft7 

The National Catholic Educational Association 
Secondary School Department . 

Suite mo, 1077 aoth St. J nw 

Washington, Df! 90007 

High school 



What is the model's pervasive orientation? 
evaluation model? 



What is the essence of the 



The model is intended to encourage renewal, excellence, and accountability. It is a 
self-study that explores (1) "The Foundation" (i.e., school philosophy, and The school 
community: A community of faith), (2) "The Diamond" (i.e., religious program, 
academic program, student activity program, program of services), and (3) "The 
Setting" (i.e., school staff, governance and finance, materials of instruction, and school 
building and equipment). 



DEFINITION OF EVALUATION USED IN THE MODEL 

How >es the model implicitly or explicitly define evaluation? 

"The self-study is a journey to the inner school .... It is within the experience of the 
student that the school does its work, and this is the interior visited by those engaged 
in the self study .... A successful self-study leads to an examination of that 
experience and reveals ways to improve its quality." 

A Steering Committee and subcommittees study "The Foundation," "The Diamond," 
and "The Setting" as described above. 
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PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

The purposes of self-evaluation are "renewal, excellence, and accountability." "Self- 
knowledge alone makes the trip worthwhile, but the real purpose of the journey is to 
discover the direction for school improvement.' 1 The process is largely summative. 



ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

The model is rigid when first used, but it can become more flexible as schools decide 
to "go on their own path." "The faculty must follow these (guidelines) closely at first, 
but as they get the feel of it they usually break free and create their own path. This 
is as it should be." 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Standards and criteria represent a synthesis of professional standards drawn from 
the Middle States Association of Schools and Colleges with the active collaboration 
of key staff and authoritative Catholic religious documents, including the National 
Catechetical Directory issued by the National Conference of Catholic Bishops. 
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MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. What is the school philosophy and the description of the school community? 

2. What are the strengths and weaknesses of programs in the areas of (a) religious 
program, (b) academic program, (c) student activity program, and (d) program of 
services? 

3. How do the following support (1) and (2) above: (a) governance and finance, (b) 
school staff, (c) materials of instruction, and (d) school building and equipment? 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

1. Description of philosophy (triangle of message, service, community leading to 
religious nature). Also student profile data (including demographics, achievement 
levels, education intentions), follow-up data on graduates, parent profile, and high 
school and broader community. 

2. Extent to which school is meeting set of questions/indicators specified in 2a-d 
above. 

3. Listings and narrative descriptions that fall into areas of administration, teaching 
personnel, support staff, professional staff survey, corporate governance, finances, 
instructional materials, library and media facilities and services, and T.hool 
building and equipment. 

MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 



Self-Study 

Visiting team referred to but duties, etc., are not described. 



MANUALS. FORMS. INSTRUCTIONS 



What is available for users and for others interested in reviewing the model? 

The National Catholic Educational Association (Secondary School Department). 
(1981. Third Printing, 1987). Self-studv for Catholic high schools (first edition) . 
Washington, DC: Author. 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

School accountability and improvement are the intent of this self-study in the areas 
of school philosophy; school community description; programs in religion, academics, 
student activities, and services; and school staff, governance and finance, materials 
and instruction, and school building and equipment. A visitation team is mentioned 
but not described in this document. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 
Origin (Sponsor) 



Standards for Quality F.lprnpntary & Middle Srhnnls 



Natirmal Agftn.Tiat.inr, nf El ementary Srhnnl Principals 

IfilK DiiWe Street 

Alexandria, VA 22314 



Level(s) 



K=8_ 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

The objective of this model "is to provide guidelines for constructive change in 
education." Two purposes are served: (1) "To give states, school districts, boards of 
education, parents, and citizens in general a means for determining the degree to 
which schools possess the ingredients for providing quality education" and (2) "To 
help individual principals assess the quality of their schools, as part of a continuing 
effort to enhance educational opportunities for their students." 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Evaluation is "process rather than product" and cannot be u d to compare tr 
performance of one school with another. The application of standards will result 
improved student/teacher performance in the classroom. This is not to replace forma, 
self-study required in regional school accreditation. The recommendation is that both 
be done. 



EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinato? Mixture? Qualitative or quantitative? Other? 

Mixture, but ivith process emphasis 

Context: Needs assessments and study of climate done 
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Input: Adequate resources and staffing are to be provided. 
Process: Primary emphasis 

Product: Student achievement addressed; some interaction is suggested. 
Qualitative analysis is primary. 



PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

Formative: Intended for improvement 



ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Very flexible 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

There were two NAESP Standards Committees "both composed of selected 
elementary and middle school educators." 

(11 in first group - 1984; 8 in second group including 3 original members - 1990) 
MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. To what extent are standards of excellence being met in the areas of organization, 
leadership, curriculum, instruction, training and development, school climate, 
evaluation and assessment? 

2. For standards where improvement efforts need to be focused, what plans are in 
place for school improvement? 



EVALUATION VARIABLES 



What indicators need to be assessed to address the evaluation questions? 

1. Quality indicators are listed for each of the 21 standards. A checklist scale is 
used: Always Evident, Usually Evident, Seldom Evident, Not Evident; also, 
comments or suggestions section for each. 

2. Identify how much improvement in a specific standard area (minimal, some, 
extensive) and assign priority to it; then plan to address it in measurable 
outcomes. 



MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data cc~ action methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

This is "a basic instrument for persons or groups interested in working to strengthen 
our nation by improving the quality of our schools." 

Assessment of meeting quality indicators for each standard (Always Evident, Usually 
Evident, Seldom Evident, Not Evident); provision for comments and suggestions. 



MANUALS. FORMS. INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

National Association of Elementary School Principals. (1990). 
Standards for quality elementary and middle schools: Kindergarten through eighth 
grade (revised edition) . Alexandria, VA: Author. 

Includes 21 standards, quality indicators for each standard, checklists for each 
standard's quality indicators, and a planning guide for school improvement. 
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MOST DISTINCTIVE FEATURE(S) 



What main points characterize this model? 

This is a process-oriented evaluation. The degree to which a school meets standards 
in the areas of organization, leadership, curriculum, instruction, training and 
development, school climate, and evaluation and assessment is assessed. A school 
improvement plan is then developed based on identified needs and priorities. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 
Origin (Sponsor) 

Level(s) 
APPROACH 



ArrrAHitfltinn Program 



NflHnnfll Independent Private Sr.hnnls Association 

(NfPSA) 

Prftsrhnnl, pigmentary, high srhnnl 



(for prnfif^ individual srhnnl 



What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

"Self-evaluation and accreditation go hand-in-hand. The object of self-evaluation is 
the improvement of the school program. Accreditation is an affirmation of the 
school's commitment of self-improvement and quality.' 1 The model involves a self- 
study followed by an on-site visitation by a team to "verify the school's findings and 
comment on the school's conclusions." 

Two levels of membership exist in this organization: "Affiliate" indicates that a 
school has received initial recognition and is committed to seeking accreditation, and 
"Accredited" indicates that "the school is substantially accomplishing its stated goals 
and objectives and has given evidence it is committed to continued growth and the 
pursuit of excellence." 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

"The school conducts a self-study and evaluation of its total program, using as a guide 
appropriate NEPSA evaluative criteria and appraisal instruments," Evaluations occur 
at 7 year intervals or when there is a change of ownership of the school. 

The school is evaluated "on the basis of the degree to which it is accomplishing the 
purposes and functions outlined in its own statement of objectives, and on the 
appropriateness of those purposes and functions for an institution of its type/ 1 In 
addition, evidence must be provided of meeting criteria in the areas of (a) general 
data, (b) philosophy and goals, (c) organization, (d) curriculum, (e) instruction and 
learning, (0 pupil services/activities, (g) school staff, (h) - rent community, (i) s iol 
plant, and (j) finance. 
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EVALUATION TYPE(S) 



Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Input: Documentation of adequate provision of instructional program, qualified 
teachers, materials, school facilities, etc., to meet the needs of all 
students at the school. 

Product: Evidence of student learning suggested in evaluation of instructional 
program area. 

Process: Used to some extent as school goes through self-evaluation and responds 
to recommendations from the Visiting Committee. 

Qualitative and quantitative data are requested. 
PURPOSE 

Is the evaluation model formative, sumroative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

The model is intended for accountability (i.e., meeting general criteria in 10 areas) 
and for improvement (ongoing school improvement in response to their own 
conclusions from the self-study and to recommendations from the Visiting 
Committee). The model is intended to be both summative and formative. 

The purposes of accreditation are "(a) to encourage school improvement through a 
program of ongoing self-study and evaluation and (b) to validate that the school has 
sound educational objectives and goals and is substantially accomplishing them." 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

"NIPSA symbolizes and promotes educational pluralism; it recognizes that schools 
which are quite different can be equally effective in providing quality learning 
programs." 
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Cri: ria in 10 areas are to be used as general guidelines during 'he self-study. The 
suggested instruments and procedures may be modified. H. >ver, the Visiting 
Committee part of the process appears to be structured inamc. uniform format. 

SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Not stated 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the eval^ *tion? 

1. To what extent is the school meeting its own goals and objectives? 

2. What evidence is there that criteria are met in the areas of general data, 
philosophy and goals, organization, curriculum, instruction and learning, pupil 
services/activities, school staff, parent community, school plant, and finance? 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

1. Review of school's philosophy, goals, and objectives and progress toward them. 

2. Documentation of meeting criteria may include copies of school documents 
planning processes, achievement data, program descriptions, etc. The school ma. 
choose how to document progress toward its goals and objectives as well as it; 
current status. 



MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team teachers), examination off financial books? 

Self-study (often including surveys), visitation committse (observations, interviews 
with proprietor, staff, students, parents), follow-up planning. 
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MANUALS, FORMS. INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

National Independent Private Schools Association (NIPSA). (1991). Accreditation 
program . Author. 

MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

Self-study and recommendations from a visitation team are to guide continued effort 
to accomplish the school's goals and objectives. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source Eyaluatira Criteria 

Origin (Sponsor) Nfltinnal StnHy nf S^hnnl Rvflliifltinn 

52Q3 Leashing Eikfi 

Ealls Church y VA 99(U1 

Level(S) EUfimgntar^ TniHHIfiVjnmnr High, RPrnnH^ry, middle 

WpI, K-12, central nfRrg, find 

mnltirnltnrflUmiiltira rial 

APPROACH 

What is the model's pervasive orientation? What is the essence of ti 
evaluation model? 

Evaluation of schools for the purpose of self-knowledge and improvement. A three 
step process of self-study, visiting team or committee, and follow-up to the evaluation. 

"Systematic process by which to assess the effectiveness of a school and to stimulate 
a school and community to establish a planned program of continuous growth so that 
its school may become progressively better." 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or e^Dlicitly define evaluation? 

Evaluation is the process of studying -hat one has, comparing it with what is 
expected, and adjusting practices to acquire desired outcomes. 



EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Model is primarily process oriented. 
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PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement t accountability, knowledge production, 
dissemination of good program practices? 

Summative with intentions for improvement. 



ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Rigid: The specific criteria, procedures, and instruments for evaluation are set 

by NSSE. 

Flexible: Adaptation of evaluation model for individual school usage. 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

National Study of School Evaluation 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

Does the school meet the standards set forth by NSSE? 

EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

N&SE standards:* philosophy and goals, curriculum, agriculture, business education, 
services, school facilities, school staff and administration, and student activities 
program (sec. level). 

* Depending on the level, these may be slightly different. 
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MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observation, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Self-study, visiting committee, and follow-up using all resources at the disposal of 
those involved. 



MANUALS, FORMS. INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

1. Evaluative Criteria for Middle Level Schools . (1990). 

2. Evaluative Criteria for the Evaluation of Secondary Schools . (1987). 

3. Evaluation Guidelines for Multicultural/Multiracial Education . (1973). 

4. K-12 School Evaluative Criteria: A Guide for School Improvement . (1983). 

5. Middle School/Junior High School Evaluative Criteria . (1989). 

6. Secondary School Evaluative Criteria . (1984). 

MOST DISTINCTIVE FEATUBE(S) 

What main points characterize this model? 

"Systematic process by which to assess the effectiveness of a school and to stimulate 
a school and community to establish a planned program of continuous growth so that 
its school may become progressively better." 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source BImp Rih hnn Schnnla- filfimpntary anH SprnnH^ry 

firhnnl RArngniH on Prngramg 

Origin (Sponsor) U S ngp»rtmpnt of F.dnrgHnr, 

Office of Erinratinnal Rpggarr h and TrnprnvprriArit 

Rprngmitinn Diyiamn 

Washington OH 9.fi9nft-fifiA* 

Level(s) Elftmpnt^ry Pmrl sArnnHpiry rA lfimPntary and g^ nn f ^ Rr y 

nrnprramfi flltema+P yPflrs^; fnr pnhlitt unH private ftohnnlQ 

APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

This is a national recognition program designed as a national school improvement 
effort. Nomination packages are prepared locally to profile individual schools. 
Nomination forms are submitted by Chief State School Officers (number of 
nominations from each to reflect the population of the state), Council for American 
Private Education, Bureau of Indian Affairs, and the Department of Defense 
Dependents Schools. A National Review Panel and site visitations are used to 
identify the Blue Ribbon schools. 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

"There are no fixed standards to be met . . . the quality of each school will be judged 
in the context of how successfully it is meeting its own goals and how well its 
programs are tailored to local needs/ The school must also "show significant 
progress in meeting state and national goals and must have attained a standard of 
overall excellence." Conditions of effective schooling (in leadership, teaching 
environment, curriculum and instruction, student environment, parental and 
community support, indicators of success, organizational vitality) guide the selection 
of schools for recognition. 

Each year two special emphases are designated (e.g., in 1991-92, history and 
mathematics are the areas that are targeted in the elementary program). 



A:19 



84 



EVALUATION TYPE(S) 



Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Context: "Schools are judged within their own context rather than in direct 
comparison with all other schools." 

Input: Must report amount of time each week devoted to English, mathematics, 
science, history, geography, resources provided; and student 
achievement. 

Process: Quantitative information is particularly used for fulfillment of eligibility 
criteria requirements in the elementary program. Quantitative data arv,* 
also used to describe the demographics of a school. Qualitative data are 
primarily used to profile the "Conditions of Effective Schooling." 



PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

"The Blue Ribbon Schools Program is designed as a national school improvement 
strategy. The intent is to affect improvement through the collaborative self- 
evaluation required of local school communities that participate and, through the 
stimulus recognition provides, to continue the pursuit of excellence. Recognized 
schools serve as models for other schools and communities seeking to provide high 
quality education for all their students." 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

™ere are three ways to meet eligibility requirements. Student achievement must 
*t a certain level, but this can be indicated in terms of actual test scores, 
percentages of student improvement, or exemplary progress and growth through a 
fully documented system of evaluation. 

A specific instrument is used by the review panel members to indicate their 
evaluations of a school's documentation. A scale of exemplary; strong, but not 
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exemplary; promising, but insufficient evidence; not exemplary; insufficient or no 
evidence is used. 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

There are no set standards. Each school is judged by a review panel according to 
progress toward meeting its own goals as well as state and national goals. 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. Does the school meet the eligibility threshold regarding student achievement? 
(elementary level) 

2. What evidence is there that the "conditions of effective schooling" are being met? 

3. What evidence of "indicators of success" are there? 

EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

1. One of the three student achievement eligibility thresholds must be met (in the 
1991-92 elementary program): 

a. "During each of the last three years, 75 percent or more of the students 
that were tested achieved at or above the 50th percentile in total 
mathematics and total reading. (Note: 6o percent of those tested is 
acceptable in any year in which there was an enrollment change of 15 
percent or more, excluding first grade or the lowest entering grade above 
kindergarten for your school.)" 

b. "During the last three years, the percentage of students who achieved 
at or above the 50th percentile in total mathematics and total reading 
increased an average of 5 percent annually. In 1990-91, 50 percent or 
more of the students achieved at or above the 50th percentile." 
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c. "The school can demonstrate exemplary progress and growth of students as a 
group as determined by a carefully worked out and fully documented system 
of evaluation." 

2. Indicators of "conditions of effective schooling" in the school in the areas of 
leadership, teaching environment, curriculum and instruction, student 
environment, parent and community support, and organizational vitality 
(elementary and secondary). 

3. "indicators of success" such as student performance measures of achievement; 
daily student and teacher attendance rates; students' postgraduation pursuits; 
school, staff, and student awards (elementary and secondary). 



MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Screening at state or other (Council for American Private Education, Bureau of 
Indian Affairs, Department of Defense Dependents Schools) level. School profile 
developed (containing information about eligibility, school characteristics, and 
conditions of effective schooling). National Review Panel. Site visitation (two to each 
site). A five-member subcommittee reviews and confirms or reverses nominations of 
whole panel. 

MANU ALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Office of Educational Research and Improvement. (1991, April). Blue ribbon schools: 
Elementary and secondary school recognition programs . Washington, DC: U.S. 
Department of Education (pamphlet). 

U.S. Department of Education. (1991). Blue ribbon schools: 1991-92 elementary 
school recognition program (nomination requirements) . Washington, DC: Author. 

U.S. Department of Education. (1990). Review panel instrument ■ Stage I . 
(Secondary School Recognition Program). Washington, DC: Author. 
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U.S. Department of Education. (1990). Secondary school recognition program 
(nomination requirements) . Washington, DC: Author. 



U.S. Department of Education. (1990). Site visit report . (Secondary School 
Recognition Program). Washington, DC: Author. 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

Individual public and private schools are nationally recognized for effectiveness in 
meeting local, state, and national goals and in educating all of their students. 
Elementary and secondary schools are recognized in alternate years. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source Profit of School F^IIah^ fPRO-S/J^ 

Origin (Sponsor) Appalflrhifl Ednrfltinnal Laboratory Tnr 

1031 Qnarriftr Street. PO Box 1348 



Charleston, WV 25325 



Level(s) Elementary, secondary ( whole districts) 

APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

The PRO-S/E is "a diagnostic tool that tells how a school system rates on factors 
research has shown to be associated with effective schools. The tool . . . uses data 
from interviews and questionnaires to assess the system's strength in each of 11 
characteristics positively associated with effective schools." Data gathering occurs 
during a 2 or 3 day staff visit to the district. A completed profile and narrative report 
are presented to the superintendent no more than 6 weeks after data are collected. 

An effective school is defined as one where (a) basic skills achievement cannot be 
predicted from students' socioeconomic status, (b) student attendance is regularly 
above 90 percent, (3) documented occurrences of vandalism and delinquency are 
relatively low, and (d) high satisfaction ratings are given the school by all 
stakeholders. 

DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Because the educational process is viewed as complex, continuous, and multifaceted, 
characteristics of school effectiveness must be considered in relation to the total 
environment and to each other. 

The degree of strength in each of 11 areas associated with effective schools is 
determined. The areas are systemwide needs assessment plan; academic objectives; 
clear understanding of superintendent, principal, teacher, and student expectations; 
classification of individual roles and responsibilities; conditions and resources for 
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learning; instructional time/task orientation; use of student assessment; system of 
rewards/reinforcement for students/staff; knowledge of school code; school climate; 
and parent support and involvement. 

EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Mixture: Context, product, inrut, process elements used. Qualitative and 
quantitative data are collected. 

PURPO SE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

The intent of this model is improvement. Both summative and formative evaluation 
techniques are used. It is also intended that districts become aware of research- 
based processes and products that will help them strengthen their weaker areas. 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Specific instrumentation is used for the surveys and interview forms. However, some 
open-ended questions are presented in the interviews, and there is some discretion 
in how a district fulfills portions of the documentation requirements. 

SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

The model was developed to assess strengths in areas that are cited in research 
literature as essential to effective schools. Pilot testing was carried out to make 
certain that the variables were, in fact, being measured by the instruments. 
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MAIN EVALUATION QUESTIONS 



What are the main questions to be answered by the evaluation? 

To what degree does the district and its schools show strength in the following areas: 
systemwide needs assessment plan; academic objectives; clear understanding of 
superintendent, principal, teacher, and student expectations; classification of 
individual roles and responsibilities; conditions and resources for learning; 
instructional time/task orientation; use of student assessment; system of 
rewards/reinforcement for students/staff; knowledge of school code; school climate; 
and parent support and involvement? 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

The 11 areas are defined on a separate "Definitions" sheet. Further descriptions from 
the literature are provided on a page entitled "Instructional Process Variables/ 1 



MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

District data form, school data form, school rating form, superintendent interview, 
principal interview, student questionnaires, and teacher questionnaires. 

The information above is collected by Appalachia Educational Laboratory staff in a 
2-3 day on-site visit to the district. 
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What is available for users and for others interested in reviewing the model? 

Appalachia Educational Laboratory. (1982). Profile of school excellence . Charleston, 
West Virginia: Author. 

Sanders, J. R., & Shively, J. E. (1984). School Effectiveness: Profile of school 
excellence. Charleston, WV: Appalachia Educational Laboratory. (ERIC 
Reproduction Service No. ED 249 228) 

Sanders, J. R., Barnette, J., & Vanco, P. (1986). A system for measuring school 
effectiveness: The profile of school excellence. Charleston, WV: Appalachia 
Educational Laboratory. (ERIC Reproduction Service No. ED 269 471) 

Sanders, J. R., Barnette, J., Osborne, A., Hange, J., & Vanco, P. (1987). The PRO-S/E 
system for assessing school effectiveness: Development, implemenation. and 
follow-up. Charleston, WV: Appalachia Educational Laboratory. (ERIC 
Reproduction Service No. ED 297 026) 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

The degree to which a district has strength in 11 areas associated with effective 
schools is determined. The process is brief, and feedback to the district occurs 
quickly. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source MSA- Sfilf-stnHy Evaluation, ArrrAHit^Hnn 

Origin (Sponsor) TV>p \TiHH1p States Association nf PAl lpgPS anrl S^hnnls 

(MSACS) 

3fi24 Market StrPPt 

Phi1aHp1phiP > 1 PA 13104 



Level(s) Elpmpntary, rmHHIp, sPcnnHary f highpr AHnratinn, sppHal 

Pdiiratinn. ar>H CnthnKc. srhonls 

APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

"Assisting member and candidate institutions in carrying out their particular 
educational goals and improving the effectiveness of each institution." According to 
the MSACS, "Modern accreditation is an activity, not a status." MSACS also believes 
that self-knowledge is greatest when the school community has a vision or concept 
of the school's future. 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

"Evaluation is a peer review conducted by a visiting team." 
Evaluation is a continual, never-ending process. 

EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
proordin&te? Mixture? Qualitative or quantitative? Other? 

The accreditation process is quantitative in nature but strives for self (School) study 
to show qualitative outcomes. 

Mixture 
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PURPOSE 



Is the evaluation model formative, stimulative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

Summative: Evaluation of whole institution, not just part, but calls for improvement 
process for the school 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Rigid: Standards of Commission 

Flexible: Each school is evaluated individually, and deviations are factored into 
the final decision. 

SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Commissions: Elementary, Secondary, and High Education; The Middle States 
Association 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

Does the school meet the standards? 

EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

Association standards relating to philosophy and objectives; educational program; 
learning media services; student services; student activities; school facilities; school 
staff and administration; and imance 
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MAIN EVALUATION METHODS USED IN THE MODEL 



What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observation, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

1. Self-study done by Steering Committee that works with the school. 

2. Visiting team to perform an objective evaluation. 

3. Action plan also done by the Steering Committee, school's improvement plan. 
MANUALS. FORMS. INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Commission on Elementary Schools 

1. The Assembly Guide for Catholic Schools . (1989). 

2. The Assembly Guide for Special Education . (1987). 

3. The Assembly Guide for the Self-study, Evaluation and Accreditation of 
Elementary/Middle Schools and School Systems . (1988). 

4. Leadership at the Summit. (1990), 
Commission on Secondary Schools 

1. Middle States Association of Colleges and Schools . (1987). 

2. Policies and Procedures Manual (1988). 

MOST DISTINCTIVE FEATURE(S) 



What main points characterize this model? 

1. Clarity, intensity, and unity are factors of quality schools. 

2. Focus developmental energies and place them in a programmatic organization. 

3. Internal capacity to control own quality and direct own improvement are 
paramount. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



lll^TLLU^iiuiiiua lUi j-ii.t.i rmi.ni.iuu 

North Central AfifinHatinn nf Srhnnls 


and PnlWeq 


Arizona State University 


Tpmnp AZ KA2ft7-3011 


Elementary, middle, ftPcrnndary; 


unit; optional and 


snerial fnnrtinn- vnratinnal and 


adult/ and ™1Wa 


preparatory schnnls 



Title or Source 
Origin (Sponsor) 

Level(s) 

APPROACH 



What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

"A school that can meet the accreditation standards of the Commission is ensured of 
having the human, fiscal, physical, and procedural resources necessary to develop a 
quality program; however, it is the responsibility of the school staff to use those 
resources cooperatively and creatively to achieve the level of excellence to which they 
aspire." NCA believes that "every school has the potentiality for excellence." 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

"Evaluation is an effort of the school to determine the degree to which its 
performance matches the prime objectives it has established for itself. The evaluation 
should generate plans for improvement and change that will make the school a better 
place for young people to grow in, to live in." 



EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

A quantitative way of evaluating a school. A mixture {ffinput/output data are used 
to determine if all standards are met. The outcome product, hopefully, is quality 
education from the school or college. 
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PURPOSE 

Is the evaluation model formative, summative, or both? Is the mode) 
intended for improvement, accountability, knowledge production 
dissemination of good program practices? 

Purpose is summative in that a determination is made of a school's accountability 
and plan of improvement to provide the best possible education. 



ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Rigid: Standards for each type or level of school accredited. 

Flexible: Each school is evaluated on its own. 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Commission on Schools, North Central Association of Colleges and Schools 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

Does the college or school meet all the standards (requirements)? 

EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

The standards included in the handbooks for each school level. 
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MAIN EVALUATION METHODS USED IN THE MODEL 



What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observation, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

School prepares report; visiting team makes observations and checks records. 
MANUALS. FORMS. INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

1- Policies and Standards for College Prepatorv Schools . (1990). 

2 - Policies a nd Standards for Elementary. Middle Level, and Secondary Sch ools 
(1990). 1 

3- Policies and Standards for Optional and Special Function Schools. ( 1990). 

4. Policies and Standards for Unit Schools. (1990). 

5. Policies and Standards for Vocational and Adult Schools . (1990). 

MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

NCA believes that if a school has the available resources and capable personnel, the 
' school has the potentiality for excellence." NCA pushes a school to resolve to be 
excellent and have a vision. 



B:10 



9 

ERJC 



99 



SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source Ontrnmpg Arirediiaiioa 

Origin (Sponsor) North CentrrJ Association Co mmission nn Srhnnlg 

Arizona State Univprsity 

Tpm P P f AZ 85282=3013 . 

Level(s) All IpvpIs (individual arhnnl hagpd, pro vision for 

several schools from nnp Hi strict. 

APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

Student success is the focus. A successful student demonstrates knowledge and skills 
in a variety of ways; the student also develops personal qualities and personal values. 
A school's focus must be on success for all students. 

This is "an outcome-oriented accreditation process focusing on student success, 
efficiency and quality-with-equity programs and requiring the school to document . 
. . the success with which the school is achieving specific learning outcomes it has 
established for itself." Both traditional and OA standards are to be met. Emphasis 
is on student outcomes, high staff expectations, and on learning by all students. 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

"OA is an accreditation option based on the school's documentation of the 
achievement of its goals and increased student success through the provision of 
quality-with-equity programs of education for all students." 

The OA process consists of a number of steps undertaken in a certain sequence. 
Establishment of committees (district coordinating committee - if several buildings 
will use the process, school OA steering committee, initial committees, self-study 
committees in target areas for study); selection of a resource specialist; collection of 
information about the school and community; development of the school's mission 
statement, belief statement, and performance outcomes; selection of target areas for 
study and improvement; series of visits by the OA team (representing the NCA); 
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selection of subgroups for study; establishment of student performance expectation 
in each of the target areas; selection of performance measures (from wide variety of 
assessment instruments and techniques); gathering baseline data and determination 
of discrepancies between current and desired levels of performance; development of 
a school improvement plan; implementation of the school improvement plan; 
documentation of student progress (student achievement, indicators of satisfaction 
with the school and its program and changes in attitudes and levels of satisfaction, 
indicators of effective-school practices, assessment of the equity aspects of the school 
program). The above is meant to be a continuous cycle. 

EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Qualitative and quantitative data are to be included. Emphasis is on process and 
product (outcomes) in this model. Context and input are addressed to some extent 
in meeting the required traditional standards for the appropriate level of school. 

PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

The model is primarily formative; improvement is its intent. "OA activities are useful 
(a) in confirming the excellence of the school in attaining certain goals and (b) in 
analyzing and reinforcing aspects of the program in which the staff believes higher 
levels of student success are possible." 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in reqmring specified criteria, 
procedures, instruments, evaluator**? 

All traditional and OA standards must be met, or documentation of the effectiveness 
of alternatives must be indicated. The individual school OA committee establishes 
specific learning goals in target areas that it has identified; there is much room for 
flexibility. There is flexibility in measurement instruments to be used as well. 
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SOURCE OF EVALUATION STANDARDS 



Who establishes evaluation standards and criteria within the model? 

The North Central Association of Colleges and Schools (Commission on Schools) has 
established standards for the type of school (traditional accreditation) and the 
standards for OA schools. 

Criteria for meeting the standards in the OA process is set by the individual school 
committee (with input from an external visitation team and internal or external 
resource specialist). 



MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. Are all our students learning? 

2. How do we know they are learning? 

3. What changes need to be made in our program so that all students will be 
successful? 

4. Is our program a quality program (e.g., Are students who come to school with 
educational disadvantages achieving at the level of their capabilities? Are 
educationally advantaged students achieving at the level of their capabilities? 
Is there evidence that teachers have high expectations for every student? Are 
resources being provided to assist students in overcoming whatever educational 
deficits they may have brought to the school)? 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

1. Indicators include documentation of meeting NCA standards tor the particular 
type of school and NCA OA standards. 

2. Indicators in the areas of student achievement, satisfaction with the school and 
its program and changes in attitudes and levels of satisfaction, effective-school 
practices, and assessments of the equity aspects of the school program are to 
be included. 
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MAIN EVALUATION METHODS USED IN THE MODEL 



What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

"The process, involves (1) planning, (2) a self-study, (3) visitations by a resource team, 
(4) implementation of an improvement plan, and (5) documentation of improved 
student learning." There is a 1-2 year preparation process before initiating the 
three/four year cycle. Beginning schools identify problems or concerns and make 
decisions regarding target areas for study and host the initial visitation by the 
resource team. Aside from "Mission and Beliefs" and "Schools and Community" self- 
study committees (or topics), most schools select three cognitive target areas and two 
affective target areas (number may vary). 



MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Commission on Schools. (1989). Outcomes accreditation: A focus on student 
success. A handbook for outcomes accreditation schools (Second Edition). 
Tempe, AZ: North Central Association of Colleges and Schools. 

Commission on Schools. (1990). Outcomes accreditation: A focus on student 
success. The OA Series. (Eleven monographs describing components of the 
outcomes accreditation model ). Tempe, AZ: North Central Association of 
Colleges and Schools. 

Commission on Schools. Outcomes accreditation: Focusing on student success , 
(pamphlet). Tempe, AZ: North Central Association of Colleges and Schools. 

Horwitz, R. J., & Marzano, R. J. (1985). A study of assessment techniques and 
underlying constructs for the NCA competencies. In Some resources available 
to help the school assess student performance in cognitive and affective areas: 
Two annotated bibliographies . Tempe, AZ: North Central Association of 
Colleges and Schools. 
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Kocher, A. T., & Belton-Kocher, E. (1985). An annotated bibliography of resources 
pertaining to the assessment of student performance in the cognitive, affective 
and psycho-motor areas. In Some resources available to help the school assess 
student performance in cognitive and affective areas: Two annotated 
bibliographies . Tempe, AZ: North Central Association of Colleges and Schools. 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

This is an outcomes model that is based on "the school's documentation of the 
achievement of its goals through the provision of quality with equity programs of 
education for all students." NCA and NCA OA standards are to be met. 



B:15 



104 



SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 
Origin (Sponsor) 



Ar.rrpriitfltinri and ftvflliifltinn 



Level(s) 



APPROACH 



Naw England Aflanniflrinn nf Srhnnla nnrl Cnllpgps 

Ths Sanborn HmisP 

IK High StTfifit : 

ffiinchfister, MA masn 

Pnhlir plpmpntary, ttmHHIp, gprnnfiflry, inHpppnHpnt gfthnnls. 

rnllpgpg, nvprflpaa, ynratinnfll, t.Pr.hnir.ft1 ( and career 

infltitiitinnfl 



What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

Self-evaluation and third party evaluation are used to assess the quality of education. 
"The goal is to maintain quality education for the entire student population." 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

"Evaluation is the process during which a school conducts a self-study and then hosts 
a Commission visiting team which evaluates the school in terms of its stated goals 
and the Commission's Standards for Accreditation." 



EVALUATION TYPE(S) 

What indicators need to be assessed to address the evaluation questions? 

Standards for accreditation relating to philosophy; curriculum and instruction; 
student services; educational media services; administration, faculty, and staff; school 
facilities; community support and involvement; school climate; assessment of 
educational progress. 
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MAIN EVALUATION METHODS USED IN THE MODEL 



What are the core data collection methods , e«g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observation, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

1. Self-study done at school by Steering Committee 

2. Visiting committee evaluation, which uses observations, interviews, 
questionnaires, etc. 



3. Follow-up progress reports 



MANUALS, FORMS, INSTRUCTIONS 



What is available for users and for others interested in reviewing the model? 



1. Accreditation Handbook. (1990). 

2 Manual for School Evaluation . (1981). 

MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

When a school has met t ae standards set forth by the accreditation, maintains those 
standards, and improves its educational program, it has taken the steps needed to 
be a quality educational environment for students. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 

Title Or Source Standards for Accreditation (IQftfl) 

Origin (Sponsor) rWirmssion on Schools 

Northwest Association of Schools and Colleges 

Roise State University 

1910 University Drive 

Boise, m 83225 

Level(s) High, junior High, elementary, special purpose, K-12 

APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

"The purposes of the Northwest Association are to: 

1. Advance the cause of education in the schools and colleges of the Northwest. 

.2. Develop educational policies and activities which will extend and improve 
educational opportunities and services. 

3. Develop criteria of evaluation which will continuously stimulate, evaluate, and 
accredit vital educational effort. 

4. Promote cooperative relationships among colleges and schools in order to attain 
these ends." 

Although the association has specific standards, it should be flexible enough to 
provide for "variations with a common framework of preconditions for quality 
education/ 1 

DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 
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An effective school provides for programs and services while providing a plan of 
continual improvement each year. A school needs to evaluate itself each year with 
a metaevaluation at least once every 10 years. 



EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Context: Annual report filled out by the school building each year. 

Process: The local school has freedom to design its own program. 

Product: Plan for improvement is required. 

Quantitative: The Association makes sure all requirements are being met. 



PURPOSE 



Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

Summative: The school as a whole is evaluated. 



Intended for accountability and improvement in each member school. 



ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Rigid: Specified procedures and criteria to follow for granting of accreditation. 

Flexible: The purpose and programs of each school are taken into account 
(deviations). 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

1988 Standards Committee 

Commission on Schools Northwest Association of Schools and Colleges 
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MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. "Minimum requirements which have been substantiated by research 
experience, or judgement of educators as basic requirements for satisfactory 
programs of education/* 

2. "Recommendations which represent desirable goals for all schools." 
EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

Association standards: Relating to Educational Program, Pupil Personnel Services, 
School Plant and Equipment, Instructional Media, Records, Quality and Improvement 
of Educational Programs, Preparation of Professional Personnel, Administration, 
Teacher Load, and Student Activities. 



MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observation, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Steps and procedures for member schools 

New Member: Must complete official application, annual report, and certify that in 
the last 3 years the school has conducted a self-evaluation using the Evaluation 
Criteria of the National Study of School Evaluation or some other means of self- 
evaluation approved by the State Accreditation Committee. Also, a visiting 
committee must review all documentation, which must also be submitted to the State 
Accreditation Committee. 

Current Member: Annual report must be completed by October 15 each year, and 
once each 10 years a self-evaluation using Evaluation Criteria must be done. 
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MANUALS. FORMS. INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

1. Standards for Accreditation: High Schools. Junior High/Middle Schools. 
Elementary Schools. Special Schools. K-12 Schools . (1988). 

MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

Standards and procedures should be sufficiently flexible to provide variations within 
a common framework of preconditions for quality education. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 

Title or Source Standards for ArrrprKtgtinn 

Origin (Sponsor) SnntWn Aggn^flHnn nf f!n1lA g Aq and SrhnnU 

iftfifi Southern i<*™ 

D^^tnr r OA 30033=4032 

Level(s) Cn\]^fi t fWiipaHnnfll gHnrarinnal institnrinng, fiP^rmHary, 

pigmentary 

APPROACH* 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

Original: To encourage standardization of school programs throughout Association 
boundaries. 

Today: To stimulate improvement, analyze a school's strengths and weaknesses, 
predict future needs, and plan educational programs to meet those 
needs. 

* Individual school is the basic unit for school improvement. 

DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

The focus is on quality through the use of self-study, visiting committee and peers, 
and the utilization of data in the implementation of programs and services for the 
total school community. 

EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Input/process: Looks at input data and their use for evalution of quality of education. 
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PURPOSE 



Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

Summative: Accountability of school is monitored and steps for improvement are 
recommended. 



ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Rigid: Standards for accreditation 

Flexible: Individuality of school is permitted in self-studies 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

.Southern Association of Colleges and Schools with input from representatives of the 
accredited schools. 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. Does the school meet the standards for accreditation? 

2. What programs and services are in place to serve the school community? 

3. What plan of improvement does the school have? 
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EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

Standards * ("principles") as they relate to purpose and philosophy, instructional 
program design, governance and organization, personnel, services, plant operations 
and facilities, finance and business operations. 

* Standards are slightly different for specific school level. 
MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observation, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

School performs a self-study, visiting committee evaluates and validates the school's 
self-study report. 

MANUALS. FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

1. Accreditation: Assuring Quality Through School Improvement , (pamphlet) 
Commission on Elementary Schools. 

2. Accreditation - The Pathway to School Improvement . (1989). Commission on 
Secondary Schools. 

3. Guide to Initial Accreditation of Schools and School Systems . (1989), 
Commission on Elementary Schools. 

4. Policies, Principles, and Standards for Schools and Accredited bv the 
Commission on Elementary Schools (For Use as a Checklist During the 1991- 
1992 School Year). 

5. Evaluative Criteria - National Study of School Evaluation 
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MOST DISTINCTIVE FEATUEE(S) 

What main points characterize this model? 

Accreditation "means that there will be a responsible balance between stability and 
innovation." 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 
Origin (Sponsor) 



Level(s) 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

Purpose is to foster excellence, encourage school improvement, and "ensure a school 
and its public that the school has clearly defined and appropriate educational goals 
and objectives." 

The primary goal of evaluation and accreditation is the self-improvement of the 
school." 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

The process of evaluation "should be concerned with the nature, scope, and effective 
teaching of individual courses, and also with their interrelationship in the 
curriculum." 

EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Process of evaluating self, visiting team evaluation, and follow-up evaluation* to 
determine whether a school has resources in place to provide services. 
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PrnrpHnrps for Appraising 

Western Association of Schools arm HollggPS 

Ififlfi Rollins Road : 

Bmlingame, r?A <umn 

F,lpmpntflry | spennHpry, continuation high school, regional 
Occupational CfintdS programs, anH nvftrgPfla schools 



PURPOSE 



Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

Summative: Self-improvement of the school through evaluation and accreditation. 
ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Rigid: Association criteria 

Flexible: Individual schools have differences that are allowed for. 
SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Western Association of Schools and Colleges 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. Does the school meet the criteria of the Association? 

2. Is there a plan for improvement? 

EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

Criteria that the Association has established for school to meet: 

1. Philosophy, Goals, and Objectives 

2. Organization 

3. Student Support Services 
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4. Cunicular Program 

5. Co-Curricular Program 

6. Staff 

7. School Plant and Physical Facilities 

8. Finance 



MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self study, site visit team, community surveys, 
observation, interviews (bour^ superintendent, principal, management 
team, teachers), examination of financial books? 

School self-study involving the entire staff and visiting committee. 



MANUALS. FORMS. INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

1. Procedures for Appraising Overseas Schools . (1989 ed.) 

.2. Procedures for Appraising Regional Occupational Centers and Programs, 
Criteria Approach . (1989 ed.) Form II. 

3. Procedures for Appraising the High School Criteria Approach Form . ( 1986 ed. ). 

4. Pursuing Excellence: Procedures for Appraising California Public High School . 
(1991 ed.). 

MOST OUTSTANDING FEATURE(S) 

What main points characterize this model? 

"The primary goal of evaluation and accreditation is the self-improvement of the 
school." 
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Profiles of State Models 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 



Alabama Pprfnrmanrp-Rflgprf Ar.rrprfitflt.inn System 



Origin (Sponsor) 



State of Alflhama Pppartmpnt nfF.rf nr.AHnr, 

Gnrrfnn Pprsnna Rlrfg " r Rnnm S03fi 

5fl N. Ripley Stlfifit 

Mnntpnmpry f AL 3fil3Q 



Level(s) 



All fsrhnnl system is unit of arxreditatiimj 



APPROACH 

What is the model's pervasive o/rientation? What is the essence of the 
evaluation model? 

Accreditation is now required of all school systems in the state. The local education 
agencies (LEAs) are measured against 21 student performance standards and 44 
system/school accountability standards (some required, some "developmental" - i.e., 
highly desirable). (There is a phase-in schedule for implementation of the standards 
and information about appropriate documentation to use; definitions of terms are also 
included as appropriate.) 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

"Accreditation is the process by which a school system is recognized as having 
attained certain prescribed performance levels, and accreditation standards specify 
the minimum performance expectations . . . Thus, an accreditation system is an 
accountability tool that represents goals valued by the public." 

The degree to which an LEA meets student performance standards, required 
system/school accountability standards, and developmental accountability standards 
is determined through annual reporting and confirmation by a visiting review team 
on a 5-year cycle. 
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EVALUATION TYPE(S) 



Is model essentially Context, Input, Process, Prcduct? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Input: Meeting of system/school accountability standards 

Product: Student performance outcomes 
Quantitative and qualitative measures used. 



PURPOSE 

Is the evaluation model formative, summative, or both? Is the mode! 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

Primarily summative (used for accountability). Some formative (used for 
improvement, to bring school systems up to " Accredited-Clear" status). 

This model is designed to (a) provide for verifying and reporting levels of compliance 
with accreditation standards; (b) provide for participation of the total school 
community in achieving required compliance; (c) serve as a vehicle for the continuous 
assessment and improvement of educational programs; (d) promote educational 
equity; (e) inform the citizens regarding the status of public education, system by 
system; (f) increase the performance level of students; and (g) engender support for 
public education. 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Quite flexible (e.g., each student performance standard can be met either through 
reaching the goal or through showing improvement). However, specific tests are to 
be used. There is more rigidity in how required accountability standards are to be 
met. 
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SOURCE OF EVALUATION STANDARDS 



Who establishes evaluation standards and criteria within the model? 

A Performance-Based Accreditation System (PBAS) Task-Force (educators and other 
interested people) working as five subcommittees drafted the standards. A Steering 
Committee (subcommittee chairs plus Task-Force Chair and Vice-chair) advised the 
State Department of Education on the development of procedures and practices 
necessary to implement the standards. Impact study was done in 1989-90 with 10 
school systems. 1990-91 was Notice Year (local officials conducted a self-assessment 
"against 11 the standards). Phase-in of standards to be completed by 1993-94. 

State Accreditation Commission appointed by State Board of Education. Ten 
members (one from each state board district and two from state at large, representing 
educators, building-level administrators, system-level administrators, training 
program professors, LEA board members, PTA members, and business personnel). 
Meets minimum of 4x/year in regular session to grant accreditation status to school 
systems and review complaints. 



MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. To what extent does the school system meet student performance standards? 

2. To what extent does the school system meet required system/school 
accountability standards? 

3. To what extent does the school system meet developmental system/school 
accountability standards? 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

Suggested documentation as evidence of compliance is included with each standard. 
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MAIN EVALUATION METHODS USED IN THE MODEL 



What are the core data collection methods, e.g., state-mandated Information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Annual report from all local education agencies; if it appears that change in 
accreditation status may be indicated, SDE staff will gather information for review 
by the State Accreditation Commission. 

On-site review by teams (on a 5 year cycle) to verify data reported annually by the 
LEA. Phase-in planned to coincide with existing special education, vocational 
education, and counseling program reviews. 



MANUALS. FORMS. INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Alabama State Department of Education. (1991, April). The Alabama 
performance-based accreditation system manual (DRAFT) . Montgomery, 
AL: Author. 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

This model is to be used for accountability and for improvement. Standards in the 
areas of student performance, required system/school accountability, and 
developmental system/school accountability are to be met. Each Alabama school 
system is required to be accredited through this process. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 

Title or Source PrngTam Quality Rpviftw (PQR) Prnrpss 

Origin (Sponsor) California Department nf WHnrat-inn 

7_i Captnl Mall 

P.O. Rot 344979. 

SflCTamftnta, CA Q4944-9790 



Level(s) Rlpmpntflry, midrilp grades, high srhnnl 

finririridiifll arhnnll 



APPROACH 



What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

The program quality review process is designed to evaluate the effects of curriculum, 
instructional methodologies, and effective strategies on students; guide the 
development of an action plan; and provide a model for a school's self-study." 

The quality criteria serve as the foundation for the school improvement processes of 
planning; implementation; self-study, and program quality review (PQR). This in 
turn leads to positive outcomes (School Performance Report). 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Planning: A school's current program is compared with the quality criteria, 
"matches and gaps" are identified, specific change initiatives are 
developed based on the findings. 

* ■ 

Self-study: After approximately 3 years of implementing the plan, a self-study of all 
aspects of the school program is again conducted in comparison with 
curricular and school-wide quality criteria. 

Program Quality Review (PQR) Process: This is conducted by an outside team, 
again identifying "matches and gaps" and possible plans for better 
alignment with the criteria. 
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Action Plans: Plans are based on formal recommendations from the outside team 
developed collaboratively by the team and the school's leadership team. 



PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

The model is both summative and formative. It is primarily intended for 
improvement. (Planning documents are considered to be working documents; changes 
are to be made as needed.) 



ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

There is much flexibility in this model. Schools are to be evaluated "against" a set 
of standards for "matches and gaps." Through self-evaluation and on-site review, 
action plans are determined. Suggestions for types of documentation to use are 
included. Decisions about what to include are made locally. 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

The standards have been established in several California State Department of 
Education documents, most notably the California curriculum frameworks. 
Department staff and field consultants have synthesized the information into 
descriptions of ideal programs. 
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MAIN EVALUATION QUESTIONS 



What are the main questions to be answered by the evaluation? 

1. Where do "matches and gaps" exist between the school's programs and the 
quality criteria? 

2. What action plans are needed to better align the school programs with the 
quality criteria? 

EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

Annual school performance reports, locally developed indicators of success, 
recommendations of the most recent program quality review, results of California 
Assessment Program and other norm-referenced tests (aggregated and disaggregated 
data), and qualitative data on student learning and the school program such as those 
listed in the Quality Criteria for High Schools. 



MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Data are obtained from the school profile, self-study, and by site visitation team 
through observations; interviews of students, staff, others; review of documents; and 
an analysis and synthesis of what students are learning and a review of any 
pertinent information. 

MANUALS. FORMS. INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

California Department of Education. (1990). Quality criteria for elementary schools: 
Planning, implementing, self-study, and program quality review. Sacramento, 
CA: Author. 
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California Department of Education. (1990). Quality criteria for middle grades: 
Planning, implementing, self-studv, and program Quality review. Sacramento, 
CA: Author. 

California Department of Education. (1990). Quality criteria for high schools: 
Planning, implementing, self-study, and program quality review . Sacramento, 
CA: Author. 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

Quality criteria (high standards) serve as the basis for school improvement efforts 
including planning, implementation of plans, self-study, and program quality review. 
These efforts, in turn, are intended to lead to positive outcomes (School Performance 
Report). Participation in the program is required for any school receiving state or 
federal categorical funding (i.e., most elementary, all middle schools, 70 percent of the 
high schools). 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source District Pm^r*** TnwarH thfi State Educa tion Goals 
Origin (Sponsor) CrinraAn nppflrt.mpnt oLEdiicatinn 

201 East CnlfflT Avptiiip 

n^nvpr r no ansna , 

Level(s) All [distinct) 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

Two items are required of every district annually: (1) results of district annual 
improvement plan and (2) executive summary (one page report in a prescribed 
format). This "can be included in annual report to public." 

(There are no state-mandated models, but school evaluation is required. Many use 
I/D/E/A model; some use North Central model.) 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Assessments of different types are encouraged (e.g., performance-based portfolios, 
writing assessments, or measures of higher order competency). There is a Colorado 
Department of Education Resource Guide identifying a variety of performance-based 
assessment strategies. 

An "effective accountability program focuses on asking the right questions, choosing 
right indicators of student performance, and reporting results to the right audience." 

"The Department of Education intends that accountability be a lool for the use of 
teachers in demonstrating credibility and responsibility." "The purpose of student 
assessment is to provide accountability to various publics while allowing students to 
guide their own learning and helping staff to prepare learning situations." 



EVALUATION TYPE(S) 



Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Mixture: Some product, context (needs assessments), process, interactive (as 
determined by district). 

Quantitative and qualitative evaluation acceptable. 

Each district develops its own accountability program tailored to its own community. 
'Colorado's public education accountability program emphasizes (1) public disclosure 
of a school's educational outcomes and (2) school improvement planning which 
significantly involves the community." 

PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

Summative: Summarizing progress toward goals each year. 
Formative: Used to develop new year's school improvement goals. 

Accountability and improvement are the primary uses. The intention of sharing good 
program practices may result from the Educational Excellence program. 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Very flexible. The only rigidity is that reports must be in a specific format. 
SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Standards are established by local school districts for the most part. The State Board 
of Education establishes statewide goals in graduation rate, attendance rate, and 
student achievement. 

C:10 



ERLC 



128 



There is an 18 member State Advisory Accountability Committee (a legally 
constructed body appointed to assist the state Board of Education in an advisory 
capacity on matters of accountability, accreditation, and policy development). 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. To what extent is the district meeting state goals in graduation rate, 
attendance rate, student achievement? 

2. To what extent is a district showing progress toward equity? 

3. What other student progress can be highlighted? 

4. To what extent is a district meeting local school improvement goals? 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

1. Outcome data (narrative and quantitative) against local goals. 

2. Narrative and/or quantitative report of progress toward equity. 

3. District demographics collected (race/ethnicity, number of schools, dropout rate, 
attendance rate, per pupil operating revenue, assessed valuation per puil, 
general fund mill levy, free lunch rate, pupil-teacher ratio), 

MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

School district profiles (demographic information), data on graduation rate, 
attendance rate, student performance, and progress toward equity. Districts select 
own methods for most part. 
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Suggestions are given in the Colorado Department of Education Resource Guide for 
a variety of performance-based assessment strategies including teacher observation, 
anecdotal assessments, interviews, participation, peer report/group evaluation, 
student products, questioning, student self-assessment, task or performance 
assessment, writing samples, etc. (See Guide in file.) Districts select from the 
suggested strategies. 



MANUALS. FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Gilbert, J. C, & Burger, P. (1990, April). Performance-based assessment resource 
guide . Denver, CO: Colorado Department of Education, (revision being done 
summer, 1991). 

Silverstein, J. E., Archuleta, E., Ellis, A. J., & Randall, W. T. (1989-90). A handbook 
for Colorado educational accountability: Community vision: Sharing 
responsibiitv for success . Denver, CO: Colorado Department of Education. 

Silverstein, J. E., Archuleta, E., Ellis, A, J., & Randall, W. T. (1990, September). 
District progress toward the state education goals . Denver, Co: Colorado 
Department of Education. 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

A cyclical process operates. School improvement goals are set in at least the areas 
of graduation rate s attendance rate, student achievement, and equity. Progress 
toward attainment of the goals is evaluated (and reported in a specified format). New 
goals are then set* There is much local autonomy. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 
Origin (Sponsor) 



Connftrtinit Srhnnl RfFArtivPTiAgg Pr^grt 
f& St.raf.Agin Sohnnl Prnfilfr Formed 



State of ConnPrtirtit DApartTriPnt of RHnraHnn 

Box 2219 

Hartford, PT 



Level(s) 



Elementary & secondary fvnlnnt.flry individual srhnnl^ 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

The State Department of Education's School Effectiveness Unit guides the school 
through the implementation of a school effectiveness process. Schools given priority 
assistance are those with a student population of at least 25 socioeconomically and/or 
educationally disadvantaged students. Technical assistance is most often given until 
there is (a) satisfactory completion of at least three action plans, (b) 
- institutionalization of the school-based planning process, and (c) significant progress 
toward mastery of the intended curriculum by all students. 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Strengths and concerns are identified through a comprehensive school assessment 
process. Faculty, parent, and student opinions about presence of school effectiveness 
characteristics are identified by questionnaire. Student achievement outcome data 
are gathered and analyzed by social class subgroups. Archival data such as school 
policies on discipline and attendance, curriculum goals and objectives, and school 
monitoring are analyzed. School improvement action plans are determined based 
upon needs identified in the assessment process. Action plans are implemented, and 
results are compiled to determine the extent to which objectives are achieved. 
Progress is reported annually, and action plans are revised. 
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Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Context: Needs assessments 
Input: Equity for all students 

Product: Outcomes 

Process: Ongoing school improvement cycle 

Quantitative data primarily collected; some qualitative analysis in assessment 
process. 



PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

The purpose of the evaluation model is to help schools institutionalize the School 
Effectiveness Process. The goal is to have a school "develop the internal capacity for 
self-renewal through continuing self-assessment, critical analysis, shared decision- 
making, and focused problem-solving*" 

The model is both formative and summative. The model is intended primarily for 
improvement but also serves an accountability function. 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

It appears that there is much flexibility throughout the process. An Action Planning 
Team is to be identified in the school; it determines the action plan based upon 
findings in the assessment. The Task Force(s) implements the action plan. School 
staff members decide who is to serve on these committees. The Department of 
Education staff members are specially trained to provide technical assistance to 
schools that decide to implement the School Effectiveness Process. A variety of 
student outcome measures is suggested. 
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SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

The individual schools set their own goals based upon findings in the assessment 
process. Suggestions for types of evidence to support student outcomes are provided 
by the Department of Education. 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. To what extent are characteristics of effective schools present? 

2. Is equity (i.e., when all students have maximum opportunities to learn 
regardless of race, gender, socioeconomic status, or prior achievement) present? 

3. How can the School Effectiveness Unit help with orientation, data 
management, planning and evaluation, resource coordination, and 
implementation assistance? 

4. Are students learning the intended curriculum? What are student outcomes? 

5. Does the school have the internal capacity for self-renewal through a 
continuing cycle of self-assessment, critical analysis, shared decision-making, 
and focused problem-solving? 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

1. Opinions of faculty, students, parents 
£ 2. Enrollment information and descriptions of program offerings and policies 



3. Assessment of where a school is with school effectiveness efforts and what help 
is needed 

4. Student outcomes 

5. Is school-based planning process institutionalized? Have several action plans 
been designed and implemented? 
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MAIN EVALUATION METHODS USED IN THE MODEL 



What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Comprehensive School Assessment Completion of Connecticut School Effectiveness 
Que, *ionnaire(s) by faculty (and students and parents where appropriate); 
studeiiv outcome data from norm-referenced and criterion-referenced tests; 
study of archival data such as school policies on discipline and attendance, 
curriculum goals and objectives, and school monitoring instruments. 

Action Planning Team Decides on action plan to meet needs identified in the 
assessment. 



MANUALS. FORMS. INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

School effectiveness unit. (1990, October). A guide to the Connecticut school 
effectiveness project . Connecticut State Department of Education: Bureau of 
School and Program Development. 

Villanova, R. M., Gauthier, W. J., Jr., Proctor, C. P., Shoemaker, J., Freedman, H. E., 
Lappert, R. E., & Waterman, H. T. (1981; revised, 1989, October). The 
Connecticut school effectiveness questionnaire . (Pilot edition) . Hartford, CT: 
Connecticut State Department of Education* 

(Strategic School Profile forms scheduled to be implemented in 1992 for use by 
districts and individual schools.) 

MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

The intent of this model is to help schools become more effective by "developing the 
internal capacity for self-renewal through continuing self-assessment, critical 
analysis, shared decision-making, and focused problem-solving." The cycle should be 
continuous, and "significant progress toward mastery of the intended curriculum by 
all students" should result. 
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ADDITIONAL INFORMATION 

Strategic School Pro&e forms are being developed for implementation in 1992. They 
will be used by districts and individual schools to report information to the public. 
Data will be provided by the Connecticut State Department of Education and by the 
local district and/or school. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 
Origin (Sponsor) 



Delawar e EffprtivP Srhnnla Prnrgsa (Draft dnpy^ 

DpIawflrA TlppartTngnt nf Pnhlir Tnat.mrt.inn 

Thg Tnwnsftnd Building 

P O Rot 1402 : 



Level(s) 



Rlgmpntary, tthHHIp IpvpI, high sr.hnnl 



TnriiviHnfll SrhnnURagpH Mnrlpl 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

This is an ongoing school improvement process (with on-site review by the State 
Department of Public Instruction once in a 5 year period [Review was mandatory in 
past 10 years; new process is voluntary, at least in 1991-92]). 

The goal of the revised process is the "establishment of new norms in a school culture 
which value information and which create corresponding practices of ongoing, 
systematic data collection and use." The goal is "school capacity-building for 
independent self-improvement." 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

"Through a process of assessment and planning, and the implementation of an annual 
site-generated school effectiveness plan, the ultimate goal of the Effective Schools 
Process will be to increase the overall level of student achievement for all students." 
This process is a "self-evaluation tool which requires the school staff to provide open 
and honest answers about the school's operation." 



C:18 



ERLC 



136 



I 

I 
I 
I 
I 
I 
I 
I 
I 
1 
1 
I 
I 
I 
1 
I 
I 
I 

ERJC 



EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Qualitative and quantitative evaluation processes are intended to be used. "The more 
traditional collection of data on school inputs and aggregate pupil outcomes will be 
replaced in this plan by information on important process interactions in the school 
and comparative outcomes for specific sub-populations of students . . ." 



PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

"This evaluation should be viewed as a formative process which looks in depth at 
what has happened in the school since the inception of the effective school process 
and examines important short range outcomes. It is a second generation assessment 
to be used by the staff for determining progress to date, making refinements and 
improvements in action strategies, and developing plans for new initiatives." 



ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

The model is flexible. There are many waj ; to show school effectiveness. Building 
teams are to prioritize needs and subsequent actions based on data. Certain 
disaggregated test score data are to be used, however. 

"The following questions are intended to guide your analysis of the correlates and 
items (questions) in the Delaware Effective School Questionnaire . While the Effective 
School planning Team is to be guided by the numerical values, the final decision as 
to the importance of any rating should depend on the collective judgement of the 
Effective School Planning Team." 
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SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Program Standards for Delaware Schools are "grounded in effective school research/' 
The Department of PI is working on the identification of "instruments and procedures 
to operationalize the attributes identified by the research and practice review." 



MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. To what extent has student achievement changed during the implementation 
period? 

2. To what extent have action plans been implemented? 

3- To what extent have the characteristics (correlates) of effective schools changed 
during the implementation period? 

4. To what extent has the implementation of action plans promoted change in 
school practices, climate and achievement? 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

Parent, student, staff opinions; state and local achievement test results; 
organizational health survey results; existence of school planning team (including 
staff and parents); existence of plan for improvement and team monitoring actions 
taken. 



MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

The new process, includes these steps: (a) di ict orientation; (b) establishment of 
Effective School Planning Team; (c) administration of written surveys; (d) analysis 
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of survey and test data; (e) action planning and implementation; (f) evaluation. 
Activities are identified for each step. 

MANUALS. FORMS. INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Delaware Department of Public Instruction. (1991). Delaware effective schools 
process (Draft) . Dover, DE. 

MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

Meeting minimum criteria for the effective school correlates or descriptors for 
organizational health factors provides "a possible means for a school to analyze its 
current conditions and to set goals." This analysis is a beginning step in school 
improvement. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 
Origin (Sponsor) 



Level(s) 



ftpm-pa Cnmprphpnfiivp Evaluation System 

QpnTgia Dppart.mpnt. of KHnr^tinn 

Offing nf Instructional Services 

Twin Towers East 

Atlanta, aA aoaa^sQAQ 

Elamantary and Secondary; 

School /District/Regional Erin rati on Services Agencies 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

The Comprehensive Evaluation System (CES) is a set of standards and procedures 
designed to evaluate Georgia schools, systems, and Regional Education Service 
Agencies (RESAs). Adherence to legal requirements is evaluated against a set of 
standards and indicators that have been in use for several years. Standards of 
quality/indicators are still being developed with implementation projected for 1993-94. 
Pilot testing of the quality standards has taken place; the Department of Education 
is waiting for funding to continue the field testing in 1991-92. Visitation teams will 
be used for this. 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Evidence is to be provided annually by each school and district to demonstrate 
compliance with legal requirements. The standards "serve as Georgia Board of 
Education's tool for measuring the extent of local systems' compliance with state laws 
and state board policy and rules." A Georgia DOE field administrator confirms the 
information through on-site visits and/or review of submitted written reports. In the 
future, quality of schools will also be assessed against standards of quality in the 
areas of student outcomes and school characteristics/climate; excellent and innovative 
practices will be identified and shared. 
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EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Current: Product (outcomes); Input; largely quantitative data 

Future: Qualitative and quantitative data will be used. Process evaluation may 
be an additional focus. 



PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

Current: Accountability is the main focus. Summative evaluation is primarily 
used. 

Future: The focus on improvement will increase in the quality standards review. 

Dissemination of good program practices and recognition of them are 
also planned. 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Specific instruments and procedures have been pilot tested. The modes of data 
collection are defined. Specific formats are defined* Suggested evidence for meeting 
standards is provided; there is some flexibility in showing compliance with some 
standards. 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Early drafts have been reviewed and revised by program personnel in the 
Department of Education, by the Department Technical Advisory Board, and by the 
State CES Advisory Committee (i.e., 35 representatives from local school systems and 
the public including superintendents, assistant superintendents, central office 
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personnel, principals, teachers, other staff, and business/community members). There 
have been many rounds of review. 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. Are legal requirements met or exceeded? 

2. Are each of the four student outcome standards being met or exceeded? 

3. Are each of the ten school characteristic standards being met or exceeded? 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

1. Documentation that legal requirements are met 

2. Student outcome data 

3. Documentation to show that effective schools standards are being met 



MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

1. Data collected by Georgia Department of Education (e.g., student assessment 
results) 

2 . Data collected during on-site visits (e.g. , observation checklists, questionnaires, 
interviews, focus groups). Twenty percent of districts to be visited each year; 
5-year cycle. 

3. Census Questionnaire (e.g., information about attendance; suspension/ 
expulsion rates; number of students on track for graduation; number enrolled 
in college preparatory, vocational, and general education programs: graduation 
rate) 
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MANUALS, FORMS, INSTRUCTIONS 



What is available for users and for others interested in reviewing the model? 

Georgia Department of Education. (1991-92). Application handbook: A complete 
guide to Georgia public school standards (system & school levels) . Atlanta, GA: 
Author. 

Georgia Department of Education. (1990-91). Pilot test standards and indicators of 
student outcomes and school characteristics . Atlanta, GA: Author. 

Georgia Department of Education. (1991-92). Public school standards . Atlanta, GA: 
Author. 

Georgia Department of Education. (1991, January). Questions and answers about the 
quality assessment component of the Comprehensive Evaluation System for 
Georgia schools, systems, and RESAs . Atlanta, GA: Author. 

Georgia Department of Education. Table 1: Indicators of quality by mode of data 
collection. 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

Legal compliance with requirements of state law are currently reported on an annual 
basis. Quality indicators in the areas of student outcomes and effectives schools 
characteristics are intended to be evaluated by 1993-94. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source Educational Assessment and Arrnnntahiiity 

Tmploman^hnn Plan f1QQO-1QQd) 

Origin (Sponsor) Hawaii State Department of Erin cation 

? O Box 236Q 

Wnnnlnln, WT QfiftfU 

Level(s) Dpyfilnping a state plan with inriivirinal srrhnnl as nnlWtirm 

unit; all Wftlg; pnhlir ftrinratinn 

APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

The intent of this model ("Context-Input-Output Analytic Model") is to have a 
comprehensive reporting system in place for the state that results in improved 
quality of public education in Hawaii (p. vi). Implementation (collection of data) will 
be at individual building level, with aggregation of information at state level. This 
uses a set of educational indicators that are linked to an analytical model. Major 
goals and objectives are presented. 

DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

There are three components to this analytical model for educational assessment and 
accountability: Context, Input, Output. A performance assessment approach is used. 

Evaluation will be based on the "idea of combining and building on selected data 
elements from existing assessment and accountability mechanisms in order tc. jroadly 
but comprehensively examine schools' performance outcomes." The intent is "not to 
integrate activities in place but rather to integrate information." 
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EVALUATION TYPE(S) 



Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Context, Input, Output 

The basic building blocks of evaluation are indicators. Indicators are statistics that 
serve as gauges to inform policymakers, educators, and the public about the 
educational system and how it is changing. It appears that both qualitative and 
quantitative data serve as indicators. 



PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

This process is intended "to provide an analytical as well as a descriptive account of 
the condition of public education in Hawaii." This is "a vehicle to provide the means 
by which educational outcomes can be examined in light of contextual factors, 
resource utilization, and educational practices and policies." 

Four uses of the information will be 

1. Accountability to the general public 

2. Accountability to policy makers and to assist in the development of educational 
policy 

3. Guide to the Department's program planning and budget development 

4. Guide to school improvement development 



ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Flexible implementation is planned at this time; the process is continually evolving. 

, . A mix of implementation strategies may be most appropriate for the Educational 
Assessment and Accountability System". 
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SOURCE OF EVALUATION STANDARDS 



Who establishes evaluation standards and criteria within the model? 

There is much difficulty in establishing standards because of the "lack of a widely 
acceptable model of educational assessment." 

There is a plan to convene a special study group on standards setting. A nationwide 
search to find experts to help will be carried out. A literature review will be done by 
the project staff. The special study group will use information found by the staff. 



MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. What are the overall condition, performance, and progress of the public 
schools? 

2. What are the accomplishments and shortcoming of the educational system? 
Where are adjustments needed? 

To what extent are intended effects, including those of major education 
initiatives/reforms, being achieved? 

EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

Indicators will be identified from existing data sources and some under development 
including School Profiles (containing multiyear statistical information for each school 
including student achievement, student demographics, student behavior, teacher 
demographics, and school-community characteristics). Other data will be collected 
from the Hawaii State Test of Essential Competencies, the College Board's Scholastic 
Aptitude Test, and the Graduate Follow-Up Survey. Additional new data sources are 
listed, and a sample set of indicators by level is provided. 
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MAIN EVALUATION METHODS USED IN THE MODEL 



What are the core data collection methods, e.g., state- mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Output: Neeu multiple measures; graduation rates, grade point average, 
performance on achievement tests 

Context: Description of community setting, family demographics, students' 
readiness for school 

Input: Course offerings, course requirements, staff allocation, teaching methods 
and finances 

Sources: School Profiles generated by Department's Information System Services 
Branch (student achievement, student demographics, student behavior, 
teacher demographics, and school-community characteristics). Also, 
Hawaii State Test on Essential Competencies, the College Board's 
Scholastic Aptitude Test, and Graduate Follow-up Survey. 



MANUALS. FORMS. INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Hawaii State Department of Education. (1989, October). Educational assessment and 
accountability: Implementation plan (1990-1994) . Honolulu, HI: Author. 
(Report submitted to the 1990 Hawaii State Legislature). 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

This is a state assessment and accountability system being developed for Rill 
implementation by 1994* Data collected at individual building levels will be 
aggregated into comprehensive state reports. Improvement in public instruction in 
Hawaii and in individual schools is intended. Data will come from current and new 
sources. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 

Title Or Source Thu nii^nia Program fnr Fiwhiat.inn, Supervision, and 

Rorngnit.inn r>f Snhnnls 

Origin (Sponsor) Tllirmis Statfi Hoard nf RHnp.flt.inn 

inn N EirsfcJStrefit ■. 

Springfield, tt, fi9777.nnm 

Level(s) Elementary, middle, and aernndary 

APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

Assuring the presence of certain required conditions and to determine if the best 
interests of the students are being met. 

DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Holding schools accountable for student performance and school improvement. 

EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

A mixture of process, compliance, requirements and outcomes, achievement, results 

On-site follow-up with continued support 
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PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

Both formative and summative: recognition for schools meeting or exceeding 
expectations. Each school building is held accountable. School improvement is the 
model's intended purpose. 



ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Rigid: State mandates (The School Code of Illinois, Evaluation, Supervision, 
and Recognition of Schools) 

Flexible: Local control of individual schools 

source of Evaluation standards 

Who establishes evaluation standards and criteria within the model? 

Illinois State Board of Education, Regulatory Process Committee 



MAIN EVALUATION QUESTIONS 



What are the main questions to be answered by the evaluation? 

Each school must meet minimum compliance in areas of teacher certification, life 
safety, board operations, class schedules, etc., but now must also provide evidence on 
the following: 



1. Are students learning? 

2. Are all students being served? 
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EVALUATION VARIABLES - 

What indicators need to be assessed to address the evaluation questions? 

1. Performance (student) 

2. Improvement (School-student) 

MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observation, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

"On-site Visit— focus on minimal compliance in areas such as teacher certification, life 
safety, board operations, class schedule, etc." 

They personally ask the question (primary concern): "How well are students 
learning?" 

MANUALS, FORMS. INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

The Illinois Program for Evaluation, Supervision, and Recognition of Schools . 
(September 1990). 

Linking Accountability to Student Performance and School Improvement in Illinois. 
(March 1991). 

MOST DISTINCTIVE FEATURE( S) 

What main points characterize this model? 

New: "Compliance alone will no longer assure recognition status," 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 



PftrfnrmflnrA-RflflPri ArrrpHitfltinn Program Manual 



Origin (Sponsor) 



TnHiflrm Hapart mpnt nf Education 

Room 229 State Hmis* 

Tnriiflnann1ia f TN 4fi9(U 



Level(s) 



All (individual arhnnlflttfllfln, srhnnl corporations! 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

"Schools may be accorded full accreditation status by meeting three requirements: 
(1) Complying with appropriate legal standards; (2) completing of a school 
improvement plan; and (3) meeting expected performance levels on student 
outcomes." 

Evaluation is used to acknowledge meeting of prescribed input standards and to 
acknowledge that a school is "an improving school or an effective school in terms of 
student performance." 

DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Legal Requirements: Compliance with Indiana statutes and administrative rules 
adopted by the Indiana Stats Board of Education that apply to each school for the 
purpose of accreditation. 

School Improvement: Development of a plan that is "an outcome-oriented document 
that is developed as a result of a comprehensive self-study conducted by the 
individual school." 

Performance Levels on Student Outcomes: "During the accreditation year, a school's 
actual performance must meet or exceed expected performance levels." The level for 
each school in each of five areas is based on "performance of schools with similar 
organizational structure, serving students of similar socioeconomic status and similar 
aptitude." 
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EVALUATION TYPE(S) 



Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Input: Monitoring compliance with legal requirements, input standards 

Context: Needs assesssment, goal setting, development of strategies in school 
effectiveness areas 

Product: Student performance outcomes; school improvement plan 

Mostly quantitative data collected. Some qualitative data are required in school 
improvement plan. 

PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

"An obvious purpose of the Performance-Based Accreditation system is to accredit 
schools in the classical sense: to grant formal approval to an institution of learning 
by a regulatory agency after the school has met specific requirements. Performance- 
Based Accreditation has a broader purpose, however, that speaks to school 
improvement." 

The goal of being accredited is not simply acknowledgement that a school has met 
prescribed input standards, but also that a school is an improving school or an 
effective school in terms of student performance." 

Summative and formative (intended for accountability and improvement). 
ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, e valuators? 

Quite rigid - most processes specified- There is some flexibility in how to go about 
development of the school improvement plan within a given framework. 
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£ What are the main questions to be answered by the evaluation? 
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SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

For legal requirements, Indiana statutes and administrative rules adopted by the 
Indiana State Board of Education. Some outside certifying agencies are used, e.g., 
Indiana State Board of Health, the Fire Prevention and Building Safety Commission, 
and Indiana Occupational Safety and Health Administration. 

Effective Schools Correlates and Performance Indicators (research literature) 

Expected performance levels: Indiana Department of Education based upon specific 
procedure 



1. Are legal standards/requirements met? 

2. Is a comprehensive school improvement plan in place, and what is it? 



I .3. Does the school meet or exceed expected performance levels in Indiana 

■ Statewide Testing for Educational Progress (ISTEP) program, proficiency in 

I, reading and language arts, proficiency in mathematics, student attendance 

I rates, and graduation rates (high schools)? 

| EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 



1. Yes/no compliance with specific legal standards and completion of multiple 
forms for multiple agencies. 

2. School improvement plan addressing nine correlate areas as well as inclusion 
of a mission statement. 

3. Test scores in ISTEP, reading and language arts, mathematics; attendance 
rates, and graduation rates. 
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MAIN EVALUATION METHODS USED IN THE MODEL 



What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

1. State-mandated reporting of compliance with legal requirements 

2. Comprehensive school improvement plan 

3. District profile of student performance on Indiana Testing for Educational 
Progress total battery scores; language arts proficiency scores; mathematics 
proficiency scores; attendance rates; graduation rates (high school), including 
degree to which district is meeting performance expectations 

4. On-site visitation from representative(s) of Indiana Department of Education 
(for schools not receiving full accreditation after submission of all written 
reports). Preliminary visitation is by Department. On-site review may follow 
(team consisting of at least one classroom teacher, one person who is not a 
classroom teacher, and one member of Department). 



MANUALS. FORMS. INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Indiana Department of Public Instruction. (1990, July). Performance-based 
accreditation program manual . Indianapolis, IN: Author. 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

Accreditation is based on (1) compliance with legal requirements; (2) an in-place 
school improvement plan that addresses nine correlate areas; and (3) meeting or 
exceeding expected performance levels in student achievement, attendance, and 
graduation rate (high school). 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 

Title Or Source Srhnnl Pgrfnmnanrg Review System 

Origin (Sponsor) Marylan d State Department ofLEldiimtinn 

200 W Baltimore Street 

Ra1timnre r MP 91901 

Level(s) All individual gnnnnlft in the atete in IQQ^ 1 Q37, and 9000 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

The School Performance Review System is one of five elements of the Maryland 
School Performance Program (MSPP). The School Performance Review is to occur 
once every 3 years on a school-by-school basis. It is intended to identify progress 
toward long-term school improvement. 

There are four parts to the program: 

1. Performance categorization of schools based on three years of school 
improvement to achieve the state data-based area standards. 

2. Publication of each school's performance category in the Maryland School 
Performance Program Report every three years. 

3. Celebrations based on level of performance category achieved and on 
outstanding growth within a performance category. 

4. Follow-up as needed. 

There are four program categories: Excellence, Success, Progress, Special Assistance. 
The MSPP is an outcome-based accountability approach. 
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DEFINITION OF EVALUATION USED IN THE MODEL 



How does the model implicitly or explicitly define evaluation? 

"The school performance review system supports continuous progress toward 
excellence by providing objective criteria for each school to (1) review its progress; (2) 
celebrate its outstanding growth and accomplishments; and (3) receive assistance if 
follow-up is needed." 

Each school is categorized by growth criteria based on the number of Excellent, 
Satisfactory, and Not Met Standards achieved on the data-based areas out of possible 
totals (i.e., 14 possible areas in high school, 9 areas in middle school, and 12 areas 
in elementary school). 



EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Product: Outcomes based 

Input and Process are included to some degree* 

Context could be included as schools examine whether local needs are being met. 



PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

To use objective criteria to measure, categorize, and celebrate a school's long-term 
growth toward MSPP excellence" 

The School Performance Review System is summative; it is intended to show long- 
term growth every 3 years* Accountability and recognition of individual schools are 
the intent. 

The School Performance Review System supports school improvement; viewed this 
way, it may also be considered formative evaluation* 
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ALLOWANCE FOR INDIVIDUAL APPROACHES 



To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

This is unclear. It appears that there may be some flexibility in how growth in the 
data-based areas is measured. Each area is to be evaluated in terms of "excellent," 
"satisfactory," or "not met." 

SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

"Standards were set by the Maryland State Board of Education for data-based areas 
reported in the annual Maryland School Performance Program Report . The Maryland 
State Board of Education will determine performance categories and growth criteria 

H 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. Which data-based area standards are being met at an excellent level? 

2. Which data-based area standards are being met at a satisfactory level? 

3. Which data-based area standards are not being met? 

4. Which schools are moving into higher performance categories or are making 
outstanding growth within the Progress and Special Assistance categories and 
should be celebrated? 

5. Which schools require or request follow-up assistance? 
EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

1. Measures as determined by the State Board of Education 
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2. Schools meeting levels as described. (Category is determined by number of 
areas being met [excellence or satisfactory] or not met) 

3. Identification of schools needing follow-up assistance based upon declining or 
'not met" performance 



MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

School profile "against" standards in data-drive areas. There is also some indication 
that other indicators may be added as implied in "Other Indicators of School 
Effectiveness." 



MANUALS. FORMS. INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Maryland School Performance Program. (1991, August). Maryland performance 
program: School performance review system . Baltimore, MD: Maryland State 
Department of Education. 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

The School Performance Review System is planned to occur every 3 years throughout 
the state of Maryland on a school-by-school basis. Performance against a set of data- 
based area standards (for high school, middle school, and elementary school) will be 
measured at thrae levels (excellence, satisfactory, not met). Schools will be identified 
for celebration based upon growth to a new performance category or within a level. 
Schools will be identified for follow-up assistance based upon declining performance 
or performance in the Special Assistance category. Categories of performance in this 
program are "Excellence," "Success," 'Progress," and "Special Assistance." Schools' 
performance categories will be published. Schools with special features (e.g., special 
education centers, alternative schools, evening high schools) may be evaluated in a 
modified way. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 

Title or Source Michigan Accreditation Progr am (MAPI 

MQftS, r P y IQQm 

Origin (Sponsor) Michigan State Poar H ofJEducatiori 

P.O. Roy 30008 

Lansing, MT 4.RQ09 

Level(s) Elfinientary acr.rpn'itat.inn standards; mirinMp anrl junio r 

high school accreditation standards 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

"The Michigan Accreditation Program is designed to provide Michigan schools with 
a common set of external standards which, when met, will assure a quality program 
of instruction leading to a consistent education for all students. The measure of that 
quality will be determined through a data-driven outcomes-based school improvement 
plan which will be developed by building staff within locally approved policies, goals, 
and objectives." 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Evaluation is comparing a school's practices against a set of standards in 17 areas. 

"To attain information for use in improving the educational program and in 
determining the need for change, the school shall carry out planned and continuous 
evaluation of the effectiveness of its program in accomplishing the objectives and 
outcomes upon which the staff and board have agreed." 
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EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

A mixture of types of evaluation are used: 

Context: School climate 

Product: Outcomes (cognitive, affective, school climate) 
Input: Staffing levels, resources 

Interactive/process: Ongoing development and evaluation with public, teachers, 

administrators. 

The model uses both qualitative and quantitative data. 
PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

The model is intended for improvement; it is both formative and summative. 
ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Rigid in staffing numbers requirements 

Overall, flexible within a set of guidelines 
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SOURCE OF EVALUATION STANDARDS 



Who establishes evaluation standards and criteria within the model? 

The standards in these documents are the state standards (established by the 
Michigan Department of Education and committees of educators working in behalf 
of the MDE). 

The criteria are partially supplied in the documents and are partially open to school 
or district interpretation. 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

Are the philosophy and goals identified? Is the school climate positive, accepting, 
safe, having an excitement about learning? Is student self-concept enhanced? Is the 
school organized to enhance the physical and social development of each student? 
Are the staffing levels adequate? Is there cooperative development of the 
instructional program? Are teachers, administrators, parents, students involved in 
development and revision of educational programs? How is pupil evaluation 
accomplished? What resources are provided? What programs are there for guidance, 
health services, pupil support services, school/community relations? Are the physical 
facilities adequate? What is the system of financial support and control? What is the 
plan for school evaluation? What school improvement activities are planned and/or 
implemented? What is the extended educational program (middle school)? What are 
the student outcomes (cognitive, affective, school climate)? 

EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

(See above) 



MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 
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Periodic self-study (students, school board members, parents, staff members, 
administrators, and community representatives). Also, ongoing evaluation through 
the budget process and internal evaluation in areas such as achievement, student 
attitudes, school climate, parents' views, and information from study of former 
students, dropouts, and staff performance. 

Visitation team 



MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Michigan Department of Education. (1990). Michigan accreditation program: 
Elementary school accreditation standards . (1988, March; revised, 1990, 
September). Lansing, MI: Author. 

Michigan Department of Education. (1990). Michigan accreditation program: Middle 
and junior high school accreditation standards . (1988, March; revised, 1990, 
(September). Lansing, MI: Author. 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

There is ongoing development of school improvement plans in this model. Teachers, 
administrators, students, school board members, parents, and community 
representatives interact to evaluate and improve schools. School improvement efforts 
are based (at least in part) on findings in the self-study and team visitation phases 
of this model. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 
Origin (Sponsor) 



RnrPflii of Arrrfidi tatinn fc S chool Improvement gjaidigS 

cbasis) - laaoflj 

University of Michigan 

School of Education 

Ann Arhor MT 4fiin9 



Level(s) 



High school 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

The mission is "to assist Michigan high schools in improving the quality of their 
schools through the efforts of voluntary accreditation and evaluation." Minimal 
standards are established that schools must meet or exceed to show commitment of 
the community and staff to provide the resources necessary for a quality high school. 

DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Evaluation is the comparison of practices against a set Gf standards provided. The 
standards are considered to be minimum standards. 



EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

This is primarily an input type of evaluation. Quantitative data on staffing, 
resources provided, and course offerings are recorded. 
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PURPOSE 



Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

Accountability is the primary focus of this model. It is primarily summative. Quality 
high schools are supposed to result from compliance with the standards. 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Rigid on human resources, curriculum, "adequate" facilities. Variances may be 
requested if there is a written purpose for them and if an evaluation component is 
in place at the outset to determine the impact of innovation on students (i.e., as part 
of a research project). 

Some standards are more open-ended, such as, "Schools shall be administered in such 
a way as to create an atmosphere conducive to effective learning and teaching." 

SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

An Accreditation Advisory Committee composed of a representative from each of 
seven educational organizations; a public representative; a U of M at-large member; 
a representative from the Bureau itself; a representative from the U of M College of 
Literature, Science and Arts; and a representative from the U of M School of 
Education. 



MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

Are policies in place to guide the school? Is a minimal curriculum provided that 
students can take? Are adequate library/media materials, space, and staff provided? 
Are there provisions for guidance, counseling, and a testing program? Are trained 
administrators and teachers in place? Are the school plant and equipment adequate? 
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Is there adequate record-keeping? Are the length and number of school days in 
compliance with the law? Is there a positive atmosphere in the school? 

EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

(See prior question) 

MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core date collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Annual report form (checklist to show district profile) - Units offered, yes/no response 
to various questions in each standard area, years of experience and education of 
administrators, number of teachers, enrollment data. 

Site visitation for new schools - after paperwork complete, visitation team scheduled 
for visit. Once every 5-7 years, a visit is made to accredited schools by a university- 
appointed accreditation consultant. (Each year 1/5 of UM accredited high schools 
[approx. 125] are visited and evaluated. Exemplary programs are identified by the 
consultants.) 



MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Accreditation standards 1990-91 . (Booklet) 

Memo to administrators seeking accreditation by The University of Michigan . From 
William J. Bushaw. 

Report to the Bureau of Accreditaiion and School Improvement Studies, The 
University of Michigan, 1990-91 . (Form) 

Sample resolution form for board of education to use . 
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The University of Michigan accredited schools 1989-1990 . 

The University of Michigan list of exemplary school programs 1989-1990 . 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

Minimal standards are included. BASIS hopes "that attainment of the standards will 
demonstrate a commitment by community and staff to provide resources necessary 
to offer a quality high school program." Input data are largely used. The hope is 
that a base is provided for further program development, experimentation, research, 
and innovation. Special note: As of June 1992, this model is no longer being 
used by the State of Michigan. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 
Origin (Sponsor) 



School A ccreditatio n Policie s, Procedures, and Standards 

Mississippi Department of Education 

Capitol Square Building 

550 f!edar Street 

St Paul UN 55101 ^ 



Level(s) 



K-19, 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

M To insure compliance with such standards and to establish procedures for the 
accreditation of public schools" 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

"ormal appraisal of educational experiences including the performance of school, 
educational programs, personnel, and student. Evaluations are conducted to 
determine strengths and areas which need improvement and may involve various 
measurements, including test scores, needs assessment, opionnaires, and Cvher 
measures which appraise the effects of the educational experiences. M 



EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Quantitative 
Input 

Product: Student outcomes 
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PURPOSE 



Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

Summative: Accountability based on performance-based accreditation 
ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Rigid: All public schools will be accredited on established standards and 

procedures. 

Flexible: Application to each school 
SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Commission on School Accreditation, Mississippi State Board of Education 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

Does the school meet the minimum standards of the Commission? 

EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

Minimum and Compulsory Standards: 

1. Active educational leadership 

2. Instructional ly focused organization 
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3. Effective instruction 

4. Change-inducing staff development 

5. Positive school climate 

6. Quality education 

7. Incentive standards 

MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

On-site accreditation audits every 5 years. These audits of the school can happen at 
any time without notification. 

MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

The Requirements of the Commission on School Accreditation: Policies, Procedures, 
and Standards . (1989). 

MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

Performance-based accreditation of all public schools 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 

Origin (Sponsor) Nphrgska T")ppflrtTriPnt. of Education 

am Centennial Mall South 

Boy 94Qft7 

T.innoln, NE 

Level(s) All (school syafafrn; p nhlir, private) 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

Nebraska schools are required to perform a comprehensive school evaluation at least 
once in a 7 year cycle. A self-study and an external team audit must be included for 
continued school approval by the State Board of Education. 

Local districts determine which model will be used. The most commonly used models 
in Nebraska are "The North Central Association Guide for School Evaluation," "The 
Self Study Guide for Catholic High Schools," "The Evaluative Criteria" model from 
the National Study of School Evaluation, and "The Nebraska Model" (an adapted 
Discrepancy Model). A staff member from the Nebraska Department of Education 
(School Management Services department) is assigned to work with the local district. 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Evaluation involves a self-study followed by site visitation by an external team. The 
self-study may be done in a traditional manner, or it may be a strategic planning 
effort toward school improvement. The latter alternative was recently approved by 
the State Board of Education. "This is consistent with our continued emnhasis on 
using outcomes as a cornerstone in the state approval and accreditation process." 
Implementation of recommendations follows the self-study and visitation. 
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EVALUATION TYPE(S) 



Is model essentially Contest, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Type of model depends upon which model was selected by the local district. The 
traditional models primarily combine input and product dimensions. Strategic 
planning for school improvement involves product (outcomes) and process; context 
and input may also be included. 

PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

"There is really only one purpose for conducting a school evaluation and that is to 
improve or maintain conditions which are most conducive to student learning/ 1 
Educational opportunities must be consistent with parental aspirations and system 
intentions. Through school evaluation the schools can determine how well they are 
doing what they say they are doing. 

The degree to which the model is formative and/or summative depends upon the 
approach used. 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

A self-study and site visitation by an external team must be used. A school system 
may select the specific evaluation model to use. 

SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

The developers of the specific model selected. (There is a legal requirement to 
evaluate schools at least once every r i years. The State Board of Education requires 
a self-study [or strategic planning effort] and visitation by an external team.) 
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MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. Are commitments to the community about provision of educational 
opportunities being met? 

2. What does the school system wish to accomplish (and what does it not wish to 
accomplish)? 

3. What evidence is acceptable as signs of success? 

4. How can school system intentions that are not being met be better attained? 
EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

Indicators depend upon model selected and school system determination. 

MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Self-study; site visit team 

(There may be other data collection methods used as well. The self-study 
requirement may be fulfilled by the development of a strategic plan for school 
improvement.) 

MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Nebraska Department of Education. Series of Pamphlets: 

Comprehensive school evaluation: Implementation of recommendations . 
(1983, October). 
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Comprehensive school evaluation: Organizing the external visitation . 

Comprehensive school evaluation: Organizing the self-study . 

Comprehensive school evaluation: The role of the external evaluator . 
Lincoln, Nebraska: Author. 

Nebraska Department of Education. (1978, revised). Nebraska school evaluation (An 
introduction to the comprehensive school evaluation process and a review of 
some essential considerations) . Lincoln, NE: School Management Services. 

MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

The state of Nebraska requires that a school system be evaluated at least once every 
7 years. A self-study and external team visitation must be included. The self-study 
may take the form of the development of a strategic plan for school improvement. 
The local school system may select the evaluation model to use. (The State 
Department provides information about advantages and disadvantages of several 
models.) 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 
Origin (Sponsor) 



School Registration Review Process 



Npw York State E diicatum Dapartmfiiit 



Albany, NY 1223Q 



Level(s) 



Ellblk filemfilltary and sfip.rmHary nndivirhial aphonic 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

Schools that have demonstrated declining student achievement over a 3-year period 
are identified to participate in the Registration Review Process. A self-study, 
visitation by a team of reviewers, and development of a Comprehensive School 
Improvement Plan (CSIP) by a local planning team occur within a specified time 
frame. 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

"A school registration review assesses the school's effectiveness and equity of 
resources at the classroom, school and district levels. The quality of the instructional 
program and the validity of the Self Study conducted by the school are also assessed. 
Department staff members assist the school in the development of an improvement 
plan." 



EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Quantitative data are primarily used in determination of need for review (i.e., scoring 
below statewide reference point on one or more of registration criteria and no 
demonstration of improvement over the past 3 years on those criteria, dropout rate 
of 10 percent or greater, lack of making annual school progress report public). 
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Qualitative and quantitative data are analyzed in the self-study and on-site 
visitation. 

The model has product and process components. 
PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

The model is both formative and summative. Improvement is a primary focus. 
ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

There is room for flexibility. Information and the process should be adapted to meet 
the unique needs of a building. The composition of the Registration Review Team 
also may vary. 

SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Comprehensive School Improvement Plan (CSIP) is developed locally. 

The Registration Review Criteria are set by the regents of the State Education 

Department. 



MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

Focus questions for the self-study and registration review follow: 

1. What current classroom characteristics and practices exist in this building for 
the area(s) of identification? 
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2. How should these classroom characteristics and practices be changed to 
improve student achievement? 

3. What current school characteristics exist in this building? 

4. How should these school characteristics and practices be changed to improve 
student achievement? 

5. What current district characteristics and practices exist? 

6. How should these district characteristics and practices be changed to improve 
student achievement? 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

School performance reports, locally developed indicators of success, improvement 
plans, norm-referenced data indicating patterns of achievement of all students as well 
as of specific groups of students, staff and parent perceptions of the school program, 
and other data as determined by the Planning Team. 

MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Self-study/district profile, site visitation team (review of records, observations in 
schools, interviews of staff and parents) 

MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

New York State Education Department (ND). Packet describing Registration 
Review Process. Albany, NY: Author. 

Appendix A: Suggested format for summarizing the results of the self-study 
Appendix B: Registration Review Visit Guide 
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MOST DISTINCTIVE FEATURE(S) 



What main points characterize this model? 

A comprehensive self-study, visitation team, and development of a Comprehensive 
School Improvement Plan are the components of this model Schools are identified 
for registration reviews when they fail to meet criteria set forth by the Board of 
Regents. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 



Program of Accreditation for Public School Units 



Origin (Sponsor) 



North Carolina Department of Pnh lic Tnstmotion 

Education Building 

llfi W Edenton Strfifit 

Raleigh, NO 21202=1112 



Level(s) 



Elementary and secondar y (public school a dministrative 
units, i ft., whole districts ; voluntary for n onpublic units) 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

Compliance with a series of state-adopted standards is the focus of this model. 
Opportunity standards (specifying acceptable educational resources and conditions) 
and performance standards (specifying satisfactory student outcomes of educational 
programs) are to be met. Self-assessment, annual reports, and internal and external 
in-depth assessment of programs and facilities occur every 5 years. 

(A voluntary program of accreditation immediately preceded the mandatory one (from 
approximately 1972-1987).) 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

"Accreditation is a process that recognizes educational institutions for establishing 
and maintaining a level of performance, integrity, and quality that entitles them to 
the confidence of the educational community and the public they serve." 

The evaluation process involves documentation to show how the standards are met. 
Assessment teams confirm the self-reports. 
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EVALUATION TYPE(S) 



Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Input: Course offerings, staffing levels 

Product: Test scores, attendance rates, dropout rates 

Process: Staff judgments regarding adequacy of services and facilities 

Quantitative and qualitative measures are used. 
PURPOSE 

s 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

"The purpose as stated is to monitor the implementation of the Basic Education 
Program" ( The Basic Education Program : General Statute 115c-12). The model is 
primarily summative, intended for accountability. 

The program is designed to 

a. Provide a uniform set of standards and guidelines for assessing 
educational effectiveness 

b. Assure the educational community and the general public that the 
state's schools maintain conditions under which satisfactory learning 
can occur 

c. Encourage local educational units to undertake continuous self-study 
, and review 

d. Identify school systems in need of assistance 

e. Provide technical and advisory assistance co such systems 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 
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It appears that there is some flexibility in how districts document compliance with 
some standards and rigidity in how they document compliance with other standards. 
Certain types of measures are indicated (e.g., test scores, attendance rates, dropout 
rates, course offerings, professional staffing levels, opinions regarding adequacy of 
services and facilities). 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Not stated. They appear to come from the North Carolina Department of Education 
and/or General Statute 115c-12 (The Basic Education Program) . 



MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. Are state opportunity standards (specifying acceptable educational resources 
and conditions) being met? 

2. Are state performance standards (specifying satisfactory student outcomes of 
educational programs) being met? 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

1. Course offerings, level of staffing, opinions about adequacy of services and 
facilities 

2. Average test scores, attendance rates, dropout rates 
MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 
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Reporting of statistics on percentages of students passing sections of the North 
Carolina Competency Tests (reading, mathematics, writing objective, writing ei^yT 
attendance rate; dropout rate; credits earned, etc. are to be included in an annual 
report; review of annual reports by DPI Accreditation Section. 

Every five years, in-depth joint assessment (DPI and local staff); analysis of reports 
by DPI Accreditation Section. 

On-site visitation team if noncompliance evident or suspected. 



MANUALS. FORMS. INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

North Carolina Department of Public Instruction. (1989, September). The North 
Carolina program of a ccreditation for public school units Raleigh NO 
Author. ' 

North Carolina Department of Public Instruction. (1991, February 19). Special notes 
on the dat a set (revised) . Raleigh, NC: Author. 

North Carolina Department of Public Instruction. (1991, February). 1990 report 
card. (Single page chart listing districts.) 

North Carolina State Board of Education. (1990). Report card: The state of schoo l 
s ystems in North Carolina (SAMPLE) . Raleigh, NC: Author. 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

Accreditation is granted when school units comply with at least 75 percent of 
performance standards and 75 percent of opportunity standards as well as with all 
state statutes, proper fiscal management, and provision of a safe and orderly 
environment for children. Accountability is the primary focus. ' 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 

Title Or Source gtanrWHsfnr Pn^^pmPnt.ary and Secondary Schools (p . 

■\a - niv oo Instnm t iniia] Program Evaluation) Orppnn 

A ^mir^gi-ra Htro T?h1pr, Jnlv 1990 

Origin (Sponsor) Oregon Department of Education 

7QO Pringlp Parkway, SE 



ftslpm, OP. ^ 7310-09.90 



Level(s) A11-1?,f,b grade and belnw 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

A self-study using a yes/no checklist against the standards in the Oregon 
Administrative Rules. School improvement is referred to in the Administrative Rules; 
but there are no supporting, descriptive documents evident. 

DEFINITION OF EVALUATION USED TN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Program evaluation is "a process for making judgments about the philosophy, goals, 
methods, materials, and outcomes of a program to guide program improvement. 

1. Identify program needs at district and school levels by 

a Annually reviewing test results and other evaluative information 
(course enrollment patterns, student attitudes, grades, etc.) 

b Cyclic program evaluations in language arts, mathematics, 
science, health, physical education, social studies, art, music 
languages, applied arts, career education, and vocational 
education (cycle must be adopted by local board of education). 

2. Develop written school and district improvement plans (staff development, 
facilities and materials, curriculum revision). 
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3. Report annually to the community on test results, school improvement. 

4. Report test results to Department of Education when requested. 

EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Emphasis on input and quantitative data collection primarily. 
PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

The model is primarily summative; accountability is tho emphasis. There is also brief 
reference to the need for school improvement in the Administrative Rules. 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

This is a fairly rigid model; yes/no answers against standards must be given. 
However, there are various ways to document the meeting of standards (e.g., board 
minutes and policies, district publications including administrative procedures, 
teacher/studentf parent handbooks, various guides, planned course statements, 
student records, records of meetings, inspection and monitoring/teacher plans/record 
books, and interviews with staff). 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Standards are established in Oregon Administrative Rules by Department of 
Education. 
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MAIN EVALUATION QUESTIONS 



What are the main questions to be answered by the evaluation? 

Is the school providing proper input in staffing; courses provided; coordinated 
instructional program throughout all grades; process for appeals/complaints/alleged 
violations of standards; course goals; graduation requirements; instruction for all 
students in infectious diseases, drug and alcohol prevention, operating policies and 
procedures; auxiliary services; extension of kindergarten programs; number of days 
of instruction; instructional tim/>; equal educational opportunity; class size; personnel; 
record keeping; guidance; he th services; emergency plans and safety programs; 
media programs? 

EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

Documentation of how each standard above is being met (i.e., mostly input data 
addressed). 



MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Checklist (yes/no) - Assurances of district school board designated Chief 
Administrative Officer, review of district materials through Department of Education 
desk audit, on-site review of practices or conditions and other methods selected by the 
Superintendent of Public Instruction. 
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MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Standards for Public Elementary and Secondary Schools (from Oregon Administrative 
Rules; Chapter 581, Division 22 - Department of Education). 

Standards Self-Study Checklist (contains all revisions as of 10/22/90) . The checklist 
is to be used by all schools; some sheets apply to only elementary or only 
secondary schools. 

MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

This is essentially an input model. A yes/no checklist is used to indicate whether 
standards are met in all areas. Supporting documentation is required. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 



Pennsylvania's F.dnratinnal Ne eds Assessment(19ft1/19ft4) 



Origin (Sponsor) 



Level(s) 



Pennsylvania Department nf F.dnrat.inn 

333 Market. S treet, Roy 91 1 

Hamsbmg, PA 1712.6-0333 : 

F.lementary; middle/jnninr high; s econdary (for use in 
district and individual srhonls within the district) 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

A self-study of a district (and individual schools in the district) is defined. 
Determination is made whei ; er criteria are met in 20-28 areas (depending on level 
of school). "Twelve Goals of Quality Education" are the bases for the documents 
generated and the process. Visitation teams are strongly suggested. There are 
separate manuals for elementary schools, middle/junior high schools, secondary 
schools, and the visiting team. 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

The guides are intended "to begin a cycle of program assessment, revision, 
implementation, and reassessment for long-range planning." 

"Securing a clear picture of the present curriculum and instructional patterns 
operating in the school" is the study. Actual practice is to be checked against "guide 
items and current national thought to determine discrepencies which can be 
diagnosed as possible school needs, depending upon the school's objectives." 
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EVALUATION TYPE(S) 



Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Context (needs assessment) and input are primary focus. Yes/No/NA checklists 
against criteria are used. 



PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

The model is intended to help schools engage in long-range planning and to involve 
themselves in a cycle of program assessments, revision, implementation, and 
reassessment. The model is both formative and summative. 



ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
. procedures, instruments, evaluators? 

The model is rigid to the extent that yes/no responses are required. It is flexible in 
how individual districts/schools meet the criteria. There are several statements that 
indicate that local situations should determine how criteria are met and which 
indicators are appropriate for that district or school. 

SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

The guides and criteria were developed by a special task force of educators appointed 
from school districts and intermediate units throughout Pennsylvania (14 people). 
Reviews by 100 other educators also took place. 

"Twelve Goals of Quality Education" were adopted by State Board of Education in 
1979. 
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MAIN EVALUATION QUESTIONS 



What are the main questions to be answered by the evaluation? 

1. Does the district/school meet criteria in the areas of central administration, 
building administration, general assessment, school environment, 
communication, mathematics, science, social studies, physical education, 
health, the arts, environmental education, practical arts, education for 
exceptional children, library media services, pupil personnel services, 
alternative patterns for learning, school-community relations, school plant? 

2. Elementary (in addition to above) - . . . meet criteria in early childhood? 

3. Middle/Junior High (in addition to 1 above) - . . . meet criteria in foreign 
language, student activities? 

4. Secondary (in addition to 1 above) - . . . meet criteria in foreign language, 
student activities, business education, driver education, home economics, 
practical arts, vocational education. 

EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

Yes/No/NA l-esponses to lists of criteria in the above areas 

MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financ 1 books? 

Self-study in each building of district; optional visitation team visit after self- study 
complete and implementation plans have begun. 
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MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Pennsylvania Department of Education. (1981, reprint 1984). Pennsylvania's 
educational needs assessment: Elementary schools self-study . Harrisburg, PA: 
Author. 

Pennsylvania Department of Education. (1981, reprint 1984). Pennsylvania's 
educational needs assessment: Middle/junior high schools self-study . 
Harrisburg, PA: Author. 

Pennsylvania Department of Education. (1981, reprint 1984). Pennsylvania's 
educational needs assessment: Secondary schools self-study . Harrisburg, PA: 
Author. 

Pennsylvania Department of Education. (1983). The visiting team guidelines for 
evaluation of Pennsylvania school districts . Harrisburg, PA: Author. 

MOST DISTINCTIVE FEATUREfS) 

What main points characterize this model? 

Self-study and optional visitation team evaluation determine the degree to which a 
district (and schools within the district) are meeting criteria in specified areas. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 
Origin (Sponsor) 



Thp Rffftpt.ivp SrVinnk Project 



Pennsylvania Dp pflrrmgnt nf Erlnrflf.inn 



333 Market Street, Roy 911 



Harrishiirg, PA 1Z12&Q333 



Level(s) 



AH (for USB in individual sc hools) 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

1. A 100-item survey instrument that can be completed in 30 minutes is available 
to help schools identify their strengths and weaknesses in the context of 
identified (effective schools) characteristics (i.e., leadership, instructional 
environment, instructional opportunity, school climate, planning, evaluation). 

2. A longitudinal study has been initiated with five districts to determine which 
practices have had an effect on student achievement as indicated by scoring 
gains and losses in the areas above. 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

School strengths and weaknesses in the context of identified characteristics of 
leadership, instructional environment, instructional opportunity, school climate, 
planning, and evaluation are determined through teacher completion of a 
questionnaire. Results are presented in terms of mean scores and percentile rank as 
well as distribution of responses for each item. 



EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? ^ 

Context, Input 
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Opinions are indicated on 4 and 5 point scales. Mean scores are determined for each 
item by school and compared against the study's mean scores. Percentile ranks for 
each characteristic are also indicated. 



PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

The purpose is to identify strength&'weaknesses in six characteristic areas associated 
with effective schools. The longitudinal study is intended to identify good program 
practices for dissemination. The model is primarily summative. 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

This model is rigid. A specific instrument is to be used, and the results are compiled 
and compared against project norms. 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Effective schools research is the basis of this model. Items from the Connecticut 
Survey (which was primarily intended for use in elementary schools) were identified 
for the initial item pool. An ad hoc teacher advisory group revised, edited, and 
developed additional items for the Pennsylvania questionnaire. The final item pool 
of 210 items was narrowed to 100 items after field testing and the use of matrix 
sampling methods. 



MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

How strong is the school in providing characteristics associated with effective schools? 
(Areas assessed are leadership, instructional environment, instructional opportunity, 
school climate, planning, evaluation.) 
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EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

Teacher opinion on items in above-named six areas 

MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Survey completion (100 items) 
MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Hojak, J. (1986, April. Reprinted 1987, June). The effective schools project . 
Harrisburg, PA: Pennsylvania Department of Education. 

MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

Teacher opinion on 100-item questionnaire indicates how strongly a school provides 
characteristics associated with effective schools. Results are compared with the 
project's overall results. Norming continues. A longitudinal study continues with the 
intent of identification of program practices that are associated with increased 
student achievement. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source Governor's A+ Award for finmrnnnity nornmitrnent to 

Excfil1enr.fi in Education (1991) 

Origin (Sponsor) Tennessee State De partmen t of Education 

Office of C ommissioner 

Nashville, TN 37243-037.5 

Level(s) All levels (individual district or collaboration of districts 

and their communities^ 



APPROAC H 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

Communities and school systems that meet 10 criteria are identified for recognition 
in this program. A community and its school system(s) can be certified for 4 years 
in this program (i.e., certification, recertification in years 1, 2, and 3); reapplication 
is then necessary. 

DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Documentation of how the specified criteria are met, review of the documentation by 
a state review committee, and site visitations to confirm what has been reported 
occur in this model. 

Although evaluation is primarily summative, attempts are made to give feedback on 
criteria that are not met. This portion of the process may be considered formative. 



EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Input: Certain types of programs are to be in place. 

Product: Evidence of academic achievement 
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Context is a factor; acknowledgement is given that there may be wide variation in 
how programs are implemented in different communities. 



A wide range of qualitative and quantitative data are to be used. 
PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

Summative. Documentation of meeting 10 criteria is to be provided. The program 
is intended to recognize school systems and communities that are using good program 
practices. 

"The Governor's A+ Award has been established to recognize communities that have 
demonstrated an exceptional commitment to providing quality education. The A+ 
Award process is designed to join families, community leaders, business and other 
interested citizens in partnerships with educators and administrators in support of 
effective schools. Recipients of the A+ Award will be recognized by the Governor's 
office with the presentation of a distinctive plaque and road sign which lets 
Tennesseans know that education is extremely important to the people of the 
community." 



ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

There is much flexibility in how the documentation of meeting the program criteria 
is done. However, the criteria themselves require that specific types of programs be 
in place. 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Not indicated. The State Department of Education administers the program. 
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MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. Do the schools/communities sponsor activities to recognize outstanding 
teachers and outstanding students? 

2. Is a formal, comprehensive plan to meet the needs of at-risk students 
implemented? 

3. Is a comprehensive adult basic education/adult literacy program offered? 

4. Is a comprehensive plan to increase family involvement in schools in 
place (including recognition for exceptional contributions)? 

5. Is there active involvement with the Tennessee Drug Free Schools 
initiative? 

6. Is there a plan in place for community involvement in schools including 
such programs as school-business partnerships, adopt-a-school program, 
local community supported student and/or teacher motivation/incentive 
programs, local education foundations, programs to increase awareness 
of the various needs of handicapped children, use of community resource 
persons in classrooms, school-based events and activities, mentoring, job 
shadowing, etc.? 

7. Is a School-Age Child Care (SACC) Program in place? 

8. Is a comprehensive vocational education program in place? 

9. Are there ongoing public information efforts to promote the 
achievements of teachers and students and successful school-related 
activities? 

10. Is there clear evidence of continuing academic achievement? 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

Several variables are suggested for each criterion. 
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MAIN EVALUATION METHODS USED IN THE MODEL 



What are the core data collection metho ds, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Documentation of meeting the program criteria including, but -not limited to, 
"newsletters, press clippings, photographs, event programs, letters, memos, 
announcements, handbooks, activities schedules, meeting minutes, pamphlets, 
brochures, survey instruments, reports, manuals, specific plan documents, video, 
audio tapes." Additional documentation (e.g., test scores) is suggested under each 
criterion. Review by a state committee and site visits by committee members are 
included in first year. 

Recertification in year 1 requires a narrative interim report, no supporting 
documentation. Recertification in year 2 requires a status report and a site visit by 
state committee members. Recertification in year 3 requires a narrative improvement 
report, no supporting documentation. Reappli cation must occur in Year 4. 

MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Tennessee Department of Education. (1991). Governor's A+ Award for Community 
Commitment to Excellence in Education . Nashville, TN: Author. 

MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

The intent of this program is to recognize communities and their schools that meet 
10 criteria (i.e., indicators of quality). How applicants document effectiveness is 
flexible. 



C:78 



SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 



TftYflfi Eduction Agcmr.y, Di vision of Accreditation 



Origin (Sponsor) 



TftYflK Stale RoarH of F,Hnrfltion 
1701 N. Hongrftsfl Avenue 



Austin, TY 7R7A1 



Level(s) 



KJ2 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

The Principles and Standards for Accreditation of Texas School Districts is concerned 
with "strengthening school districts. " . . . Accreditation visits became technical 
assistance visits." 



DEFINITION OF EVALUATION USED IN THE MODEL 

m How does the model implicitly or explicitly define evaluation? 

In 1984, the primary focus of accreditation changed from district compliance to the 
principles of monitoring the quality and effectiveness of a district's instructional 
program. 



EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Product: Evaluating student outcomes 



PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 
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Summative: Student outcomes intended for improvement of school districts and for 
the accountability of those districts. 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Rigid: Five correlates of effective schools 

Flexible: All other aspects of the accreditation are very flexible. "The intent is 
not to dictate what to do in all situations, but to detail procedures that 
have been found to work in most situations." 

SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Texas State Board of Education 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. Does the district have the five correlates of effective schools in place? 

2. Does the district meet student, campus, and district performance levels? 

3. Is there a long-range plan for education in the district? 
EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

Five correlates: 

1. Instructional leadership 

2. School climate 

3. Teacher behavior/high expectations 

4. Measurement 

o. Instructional focus 
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MAIN EVALUATION METHODS USED IN THE MODEL 



What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Accreditation team visit to determine if the district meets all procedures and 
standards. 



MANUALS. FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Texas Education Agency, Division of Accreditation: Operational Procedures Manual . 
(1989). 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

"Improve the statewide accreditation process by using a performance-based 
accountability and evaluation system and attend, on a priority basis, to those districts 
most in need of regulatory attention." 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source __ 

Origin (Sponsor) Wisconsin Dpparf.m ent of Pnhlir. Tnsf-.rnr.t-.inn 

19.5 S Wfthster 

M adison, WT 53707 

Level(s) KM 9. SrwHal Schools 

APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

Checklist against standards for the state 

DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Evaluation: Determining whether a district is meeting each standard (yes/no) 

Evaluation of all certificated personnel - "a systematic procedure to measure the 
performance of licensed school personnel" 

EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Input, primarily; some product 
Data are largely quantitative. 

PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 
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This model is intended for accountability. It is largely summative. This is basically 
an input model and offers little assistance to districts toward improvement beyond 
compliance with minimal standards. 



ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

The model is quite rigid. Yes/no responses are needed for each standard and specific 
"District to Provide" documents are specified. 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Wisconsin Department of Education and Legislature 



MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

Is the district providing schedule, staffing, materials, facilities, etc., to meet the 
Wisconsin "Standards and Rules" requirements? 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

The district is to provide documentation for every item. Examples are lists of 
staffing, schedules, courses of instruction, daily time in schools, 3-5 year facilities 
plan, professional development plan, etc. A report to district residents is also 
required. 



MAIN EVALUATION METHODS USED IN THE MODEL 

Wh^t are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community 
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surveys,observations, interviews (board, superintendent, principal, 
management team, teachers), examination of financial books? 



MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

School district standards and rules . (1991, April). (Wisconsin Statutes and Rules.) 

School district standards review 1990-1991 (draft) . Checklists in areas of staff 
certification, staff development, remedial reading, 5-year old kindergarten program, 
all subject areas. 

MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

Evaluation is the determinat : on of whether state standards are being met by a 
district. 
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"To ensure compliance with the standards, the state superintendent shall annually I 
conduct a general on-site audit of at least 20 percent of all school districts." They are ■ 
selected by stratified, random sample. "The audit process must involve school board 
members, school district administrators, teachers, pupils, parents of pupils, and other B 
residents of the school district." Districts must be notified 90 days before an audit. 
They are to compile documentation to support responses in the "School District 
Standards Review" yes/no checklist. ■ 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source School Evaluation 



Origin (Sponsor) Ann Arhor Pnhlip Schools 



2555 S Slate Street, Box 1188 



Ann Arhnr, MT 481 OR 



Level(s) Elementary, middl e school, high school (individual school 

district) 

APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

Ongoing school improvement efforts are guided by context evaluation; Board mission, 
goals, and focus objectives; student performance indicators including aggregated and 
disaggregated data; continuous curriculum development; and Michigan Public Act 25 
requirements. Increased participation by staff, students, parents, and community 
members in the constant cycle of planning, implementation of change, and evaluation 
is encouraged by focus objectives in the areas of site-based leadership, strategic 
planning, outcome-based education (using local curricular objectives), multicultural 
education, and transition to middle school and four year high school. 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Evaluation in this district occurs in many ways. Evaluation of context, input, 
processes, and outcomes (products) appears to be continuous. Evaluation serves the 
functions of guiding goal-setting and decision-making as well as providing 
accountability information to the internal and external audiences. 

"The goal of Research Services is to help district staff become active seekers and 
users of information for making education decisions." 
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EVALUATION TYPE(S) 



Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Context: Attitudes and Opinions of Voters Toward the Ann Arbor Schools and 
strategic planning 

Product: Student outcomes 

Process: Ongoing school improvement and strategic planning efforts 

Input: Provision of curriculum that uses developmentally appropriate 

instruction and alternative assessment strategies based on significant 
outcomes for each student. 



Information is qualitative and quantitative. 



PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

Evaluation is both formative and summative. Improvement and accountability are 
the major emphases. In the evaluation processes that this district uses, it also 
appears that knowledge production and dissemination of good program practices 
occur. Shared decision making based upon a wide range of information would seem 
to support all four intentions listed above. 

"The purposes of assessment are diagnosis, placement, instructional feedback, 
grading, general achievement, disposition, and program evaluation." Multiple 
assessment of students' work and programs should occur. 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Staff, parents, students, and community members are to be involved in school 
improvement/strategic planning efforts. There is quite a bit of flexibility in how some 
evaluation takes place. However, there are also specific key indicators that must be 
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reported (e.g., demographics and Michigan Educational Assessment Program [MEAP] 
scores for elementary, middle, and secondary schools; California Achievement Test 
and Degrees of Reading Power Test for elementary and middle school; rates of 
participation for middle school and high school; ACT scores, grade point averages, 
graduation rates, dropout rates, and course enrollments for high schools). 

SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

The Board of Education establishes the mission, goals, and focus objectives for the 
district. Strategic planning takes place at both the individual building and district 
levels. Much of the curricular, instructional, and assessment practices of the district 
ap r ear to be guided by educational research. 

Standards to gauge effectiveness in the areas of multicultural education, 
learning/thinking skills, citizenship and ethical behavior, use of technology, 
communication skills, and writing across the curriculum were developed by staff 
committees. Standards are defined as approved models for common understanding 
and the bases for judgment of effectiveness. These are to apply in all curricular 
areas. 



MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. What are the needs of the students as perceived by various groups (e.g., public, 
staff, parents, students)? 

2. Is the curriculum aligned to best meet student needs? 

3. Are excellence and equity issues being addressed? 

4. What progress is being made toward district and individual building school 
improvement and strategic plans? 

5. What changes need to be made in school improvement plans for subsequent 
years? 

6. Are Public Act 25 requirements being met? 

7. Are curriculum standards being met? 
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EVALUATION VARIABLES 



What indicators need to be assessed to address the evaluation questions? 

Public opinion surveys, ongoing discussions by school improvement and strategic 
planning teams, documentation of efforts toward curriculum development and 
alignment, annual reports to the community (per PA 25 requirements), study of 
programs against identified standards. 

MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books)? 

District and school profiles, committee work (school improvement, strategic planning, 
curriculum development, etc.). See also "Allowance for Individual Approaches" 
section. 

MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Ann Arbor Public Schools Documents: 

"An invitation to help develop action/implementation plans for the Ann Arbor 
Public Schools strategic plan" (brochure, 1991). 

Annual report of the Ann Arbor Public Schools (in accordance with Public Act 
25) . (1991, May). 

Board of education agenda meeting of September 4, 1991 (including "Key 
Indicators" and documenting student growth [p. 50] and primary assessment 
try out [p. 56]) 

Curriculum yellow pages . (1989-91). 

"Documenting student growth through on-going classroom Assessment." (The 
implementation year 1990-91) 

Effective classroom practices (1986, June) 
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"Excellence with equity: A five-year plan to realize the highest possible 
potential for all students (equity) while extending current standards 
(excellence)" (1987-88, 1991-92). 

Figure I - Curricular and instructional accountability chart 

Huron High School sample disaggregated course grades (including attendance 
and grade point average analysis) (1990-91, first marking period) 

Middle schoo] evaluation materials 

Research Services draft mission and philosophy 

Sample elementary school profile (Carpenter School, 1990-91) 

Sample high school profile (Huron High School, 1990-91) 

Sample middle school profile (Forsythe Middle School, 1990-91) 

Project Outreach. (1990, December). Attitudes and opinions of voters toward 
the Ann Arbor Public Schools: A public opinion survey of registered 
voters . Lansing, MI: Michigan State Board of Education, Michigan 
Department of Education. 



MOST DISTINCTIVE FEATUKE(S) 

What main points characterize this model? 

Ongoing evaluation guides the initiatives of this district (including school 
improvement efforts, strategic planning, building level decision making, curriculum 
development, etc.). A wide variety of information is used with a particular focus on 
context evaluation and outcome data. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 

Title or Source 



Origin (Sponsor) Charleston County vSr.honl District 

3 Chi sol m Street 

Charleston , SC 29401 

Level(s) Secondary (HistrirAwirift data mller.tion hy srhooi^; a 

bit elementary (graclo 

APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

Data are collected and analyzed for this 44,000 student, 70 school district in two 
areas: percentage of students meeting high school requirements at the appropriate 
age and percentage of students meeting CCSD standards for high school graduation 
within six years of entry into grade 9. These goals are targeted for intervention in 
the district's school reform efforts. Also examined are demographic data and factors 
that detract from learning. 

DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Evaluation is the calculation of the following: 

1. Percentage of students meeting CCSD high school graduation requirements at 
the appropriate age 

2. Percentage of students meeting CCSD standards for high school graduation 
within six years of entry into grade 9 

Calculation methods (and problems with them) are described. 

Data are also collected on suspensions; disciplinary problems via teacher ratings of 
student behavior; attendance rates; drug and alcohol use (in survey); and girls 
dropping out of school due to pregnancy. 
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CCSD Teacher Assessment Program (TAP) to assess performance of teachers 
providing instruction to high risk students. 



EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other?- 

Product; Quantitative; Some Context (in survey information) 
PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

To monitor progress toward district reform (intervention) goals. 

Summative 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Rigid: Specific information required in specific format. 
SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Not clear, but it appears that the district does this. 
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MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. Is progress being made toward the goal of increasing the percentage of 
students meeting CCSD high school graduation requirements at the 
appropriate age? 

2. Is progress being made toward the goal of increasing the percentage of 
students meeting CCSD standards for high school graduation within six years 
of entry into grade 9? 

3. What are factors that may be hindering student achievement? 
EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

Indicators are percentages for 1 and 2 above; demographic data from school reports 
and surveys 

MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profiie, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Annual data from each high school on percentage of students meeting CCSD high 
school graduation requirements at the appropriate age (defined as percentage of new 
first graders enrolled in the fall who graduate from high school by the end of the 
summer of their twelfth year) and percentage of students meeting CCSD standards 
for high school graduation within six years of entry into grade 9, i.e., "the number of 
rising eighth graders who graduated in four, five, or six years (spring or summer)." 

Some surveys 
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MANUALS. FORMS, INSTRUCTIONS 



What is available for users and for others interested in reviewing the model? 

Rose, J., Godin, D., Gottfredson, D., & Gottfredson, G. (1990, January). 2001: 
A vision for the future - A plan for monitoring CCSD's progress . 
Charleston, SC: Charleston County School District. 

Rose, J., & Godin, D. Merging traditional evaluation and research functions . 
Charleston, SC: Charleston County School District. 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

This is a plan for monitoring the district's progress toward reform goals with no 
increase in evaluation staff or budget. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source Strategic Planning Model for School Improvement and 

Arwmntflhility 

Origin (Sponsor) Cb *r\ Qtte=Mfick3 on burg School s 

1501 Euclid Avenue 

Chariots, NC 28203 

Level(s) Elomfilltary, secondary (to He implemented in individual 

Vmildings in district) 

APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

"The Comprehensive School Planning Process is a school-level planning process 
designed to involve all members of the school community in decision-making." The 
Board's mission statement includes goals in the areas of student attendance, student 
achievement, students in higher education or work, and parent and volunteer 
participation. A district school improvement plan is required by state law. The plan 
has two components: the system component ("waivers from state regulations and/or 
law and the system's expectations for improvement over the next 3 years") and the 
individual component ("a career ladder program which provides bonuses for teachers 
and administrators who are deemed outstanding by a series of observations"). 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

System and individual performances are measured against standards of 
accountability. 

Each school is required to have a school improvement plan. About half of the schools 
are "generating individual school plans and assessing their performance using a list 
of indicators similar to the outcomes which will determine the system's performance. 
The other schools are developing comprehensive school plans using the five effective 
schools' correlates as a framework for assessment and planning." Parent, staff, and 
student attitudes and performance perceptions are determined on surveys, and 
performance on the system accountability standards are also tracked. 
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Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Product: Outcomes, in all options 

Context: Attitude and perception surveys, in half of schools 
Process: Ongoing school improvement 

PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

The model is both summative and formative. It is intended for accountability and 
improvement. Because individual schools are approaching school improvement in 
either of two ways, some knowledge production about each approach should result. 
Good program practices should be identified in the bonus awards process for school 
or work setting performance. 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

The model appears to be flexible in that schools develop their own improvement plans 
and then measure progress toward them. However, the system goals and objectives 
call for specific data to be collected: student attendance (percentage of eligible 
students); graduation rate; student achievement in reading, writing, competency tests 
(10th grade), SAT scores, Advanced Placement examinations; percent of graduates 
going on to work or higher education; parent attendance at parent-teacher 
conferences; and number of volunteers. 

Selection of the Comprehensive School Planning Team has flexible criteria, but must 
include the principal and at least three parent representatives. Representatives from 
grade levels, departments, support staff, students, and noninstructional staff are 
suggested. 
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SOURCE OF EVALUATION STANDARDS 



Who establishes evaluation standards and criteria within the model? 

Two sources for system standards of accountability: (1) Performance indicators from 
the state accreditation process for school systems and (2) Charlotte-Mecklenburg 
Board of Education adopted goals and measurable objectives. The measurable school 
objectives resulting from the school-level planning process are another standard of 
accountability in the planning model. 

Standards of accountability for the individual component of the district plan are the 
appropriate state performance appraisal instruments. Guidelines for the bonus 
awards process have been developed by the system. 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. What are current performance levels for this school in student atcendance and 
graduation rate, student achievement, dropout rate, number of students going 
into higher education or work, parent/volunteer participation, and other areas 
designated in the school improvement plan? 

2. What are the mission statement and long-range goals for the school? 

3. What are the highest priority goals and measurable objectives for the coming 
year? 

4. What areas are placed in a maintenance plan (i.e., areas of strength to be 
maintained)? 

5. What progress has been made at the end of the year? 
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EVALUATION VARIABLES 



What indicators need to be assessed to address the evaluation questions? 

1. Assessment Information Document (performance indicators) 

2. Three-Year Comprehensive School Plan 

3. Annual School Growth Plan 

4. Maintenance Plan 

5. Summary of Results (End-of-Year Report) 

MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

District profile, individual school profile, development of individual school 
.Comprehensive School plans, ongoing monitoring and adjusting of implementation 
process (in 1989-90 pilot and continuing). 

"We will continuously study the implementation process-looking for needed revisions 
and, more importantly, identifying the support needed by professionals who are 
responsible for its implementation." 

MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Charlotte-Mecklenburg School System. (1990-91). Charlotte-Mecklenburg Schools 
strategic planning model for school improvement and accountability . 
Charlotte, NC: Author, (brochure) 

Charlotte-Mecklenburg School System. (1990-91). The comprehensive school 
planning process: School improvement - one at a time - together. 
Charlotte, NC: Author. 

Charlotte-Mecklenburg School System. (1991, April). Proposed school 

improvement and accountability plan 1991-92 through 1993-94 . 
Charlotte, NC: Author. (Describes proposed differentiated pay plan, 
modifications of Individual Incentive Program, alternative model for 
performance evaluation, and plan for awarding bonuses to individuals 
based upon the performance of their school or work setting.) 
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MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

This model is an individual school model that focuses on school improvement through 
a comprehensive school planning process. Student outcomes (systemwide and in 
individual schools) and individual staff performance components are evaluated. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 



Strategic Planning. Sr.hnnl Tmprnvftmpnt, anH 

Accountability MnHpl 



Origin (Sponsor) 



Pes Moines Public Sr.hnols 



1800 Grand AvfimiP 

Dfis Moines, TA 50302=3382 



Level(s) 



Eleme ntary. spcnnHary (jndisddua] school; district; 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

Strategic planning and school improvement efforts characterize this model Ongoing 
evaluation of individual schools and district- wide curriculum and instruction occurs. 
Individual classroom and professional support services staff members are provided 
with a research-based framework for best serving the needs of students. 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Evaluation appears to be included in all aspects of this district's operation. 
Effectiveness, equity, and efficiency of district programs are monitored through a 
program evaluation model. "Program evaluation is the process of delineating, 
obtaining, and providing useful information for judging decision alternatives." The 
CIPP model is used as indicated below: 

Context evaluation: Describes the actual and desired conditions surrounding 
a program. Current program status, unmet program needs and opportunities, 
and problems preventing needs from being met and opportunities from being 
pursued are identified. 

Input evaluation: Seeks useful information for determining how to allocate 
limited resources to meet established or intended goals and objectives. The 
tying together of district structures (e.g., programs or services) with district 
resources (e.g., money, staff time, equipment and facilities) is integral. 



D:15 



218 



Process evaluation: Detects differences between what was planned to occur 
and what is actually occurring; alteration of courses of action to meet the 
intent of the original design; documentation of events and procedures by 
recording what is occurring in relation to the planned course of action. 

Product evaluation: Measures and interprets program outcomes from the 
perspective of the planned program goals and objectives as set forth in the 
context evaluation segment of the process. 

" Strategic planning is a process and a discipline for facilitating the application of 
limited resources to competing needs in the context of a complex, changing 
organizational environment." Context and input evaluation are a large part of 
strategic planning. 

EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

See prior section. Qualitative and quantitative data are used. 
PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

This model is both formative and summative. It is intended for improvement and 
accountability. 

District Mission Statement: "The Des Moines Independent Community School 
District will provide a quality educational program to a diverse community of 
students where all are expected to learn." 
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ALLOWANCE FOR INDIVIDUAL APPROACHES 



To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Program evaluation is to be used. Certain types of data are collected by the district's 
Department of Information Management. There is flexibility in how evaluation is 
carried out by school-based management teams. 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

The district improvement plan has been developed by a district team. Other 
documents have been developed by other teams. "The district encourages initiative 
and planned change through the involvement of stakeholders." Additional building 
evaluation criteria are determined by the building management teams. 

' MAIN EVALUATION QUESTIONS 
What are the main questions to be answered by the evaluation? 

1. What is the status of the current program(s) under review? What needs and 
opportunities have not yet been met? What is preventing accomplishment of 
the intended program goals (context evaluation)? 

2. Are district resources being used to best accomplish intended goals and 
objectives (input evaluation)? 

3. What is occurring in relation to the planned goals and objectives? What 
changes need to occur for the intended goals and objectives to be accomplished 
(process evaluation)? 

4. What are the outcomes of the program(s) (product evaluation)? 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

1. Staff opinions of goal accomplishment, remaining needs, factors blocking 
accomplishment. (Student and community opinions may also be sought.) 
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2. Reports of actual resource allocations to different programs. Review of 
recommendations by building or district teams about what needs to be 
accomplished and how this might best be done. 

3. Information from 1 and 2 above. Decisions about changes to implement. 
Building and district staff interaction is important. 

4. Outcome measures, such as student academic performance (test scores and 
portfolios); demographic data; other, as determined by the building or district 
improvement teams. 

MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

District/building/program profile information; progress against identified goals, 
objectives and action plans in the district/bnilding/program improvement plans. 
Evaluation is to be built in to all strategic plans. 

MANUALS, FORMS. INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Restructuring documents dealing with district, school, and classroom improvement 
and accountability issues are 

District Focus 

Strategic Plan for 1989-91; District Improvement Plan for 1991-1992; 
1990-91 Withdrawal Report; Program Evaluation (CIPP) model; 
Program Evaluation - Reading K-12 and Language Arts K-5 
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School Focus 



Schocl-Based Management (paper); Board Policy 330: Team 
Management; School-Based Management through Shared Decision 
Making (organizational plan); School Information Base (high school 
illustration); Framework for Effective Support Services 

Classroom Focus 

Framework for Effective Teaching 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

Context, input, process, and product evaluations of programs are used to gather 
information for shared decision making by building-based management teams as well 
as district teams. Strategic planning efforts are operating throughout the district. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 



On-Site Sr.hnnT AsSfiSSIDfint "Plan f/>nH F,H ) 



Origin (Sponsor) 



District of Columbia Eiihlic Schools 

TnHf-ppn^PnrP Avpmnft r RE 

Washington, DC, 20003 



Level(s): 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

This model provides the plan for successfully carrying ou, the assessment of each 
school by providing every administrative level from the superintendent to the 
individual building principal with a means of assessing a building" s performance and 
improving the school's future. 

DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

This model's objective "is to promote excellence by providing a viable and 
comprehensive instructional program." The plan will give "... continued success, 
assessment, and improvement . . ." that lead ". . . to the attainment of knowledge, 
competencies, and skills." 



EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Context: Instructional program, learning climate 
Input: Documentation provided by principal 
Process: Primary emphasis 
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Product: Some student achievement is addressed. 

Qualitative analysis is primary. 

On-site follow-up with continued support. 

PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement , accountability, knowledge production, 
dissemination of good program practices 

Formative and summative: Intended for ". . . seeking greater improvement." 
ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
, procedures, instruments, evaluators? 

Very rigid: Guidelines to be followed 
Flexible: Individuality of evaluators 

SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

1986: Division of Quality Assurance and Management Planning, presently the Office 
of Educational Accountability and Planning 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. To what extent is the school being assessed meeting expectations in the areas 
of instructional program, management, student attendance, learning climate, 
parent and community involvement? 

2. What plans are in place for future improvement? 
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EVALUATION VARIABLES 



What indicators need to be assessed to address the evaluation questions? 

For each expectation area a rating scale is used: below expectation, meeting 
expectation, exceeding expectation, and NA; there is also a documentation or 
comments section for each. 



MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observation, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

1. Principal fills out self-study. 

2. On-site assessment team visit, which includes observations, interviews with 
principal, classroom visitations. Assessment team must be familiar with 
district and local school educational plans. 

3. Each school is to be assessed on a rotating basis at least once every 5 years in 
order of greatest need first. 

MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

1. On-Site School Assessment Plan . (1988, November). 



MOST DISTINCTIVE FEATURE (S) 

What main points characterize this model? 

Improvement through assessment of instructional program, management, student 
attendance, learning climate, and parent and community involvement. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 



Assessment Program (Conso rtium Mo^p^ 



Origin (Sponsor) 



Northeastern Educational Intermediate Unit 

Old Plank RoaH 

Mayfjeld, PA 



Level(s) 



ElpmPmtflry; secondary (mriivirhial srhnnn 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

This model is intended for improvement. School effectiveness questionnaires and a 
visiting team review of criteria based on effective schools characteristics are 
completed. 

DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Staff members complete a 100 item school effectiveness questionnaire (elementary or 
secondary) in which perceptions about presence of effective school factors are 
measured. Identified areas are safe/orderly environment, clear school mission, 
instructional leadership/decision making, high expectations, opportunity to learn and 
student time on task, frequent monitoring of student progress, and home-school 
relations. 

A trained visitation team then assesses curriculum, instruction, and staff 
development. Guiding questions are given in the areas of curriculum organization; 
curriculum commitment to school district goals; instructional activities; student 
assessment program; physical learning facilities; curriculum development; climate for 
learning; administration and supervision of curriculum, instruction, and staff 
development; classroom and library instructional resources; guiding student learning; 
and subject area criteria. A match of questions between characteristics of effective 
schools and assessment areas is also provided. 
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EVALU TION TYPE(S) 



Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Context: Effectiveness surveys for staff 

Input: Extent to which planning and performance indicators guide personnel 
and other resources; compliance with legal requirements 

Product: Student performance measures used 

Qualitative analysis is used extensively (e.g., direct observation in classrooms; 
interviews with staff and community members, etc.). Quantitative data are also used 
in portions of the assessment process. 

PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

This model is primarily formative and intended for improvement. Some summative 
information is used to guide the improvement efforts. 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Many of the guiding questions for the visitation team are broad. It appears that 
there is flexibility in how visitation teams assess many of the identified areas of a 
particular school. Yes/no indications of compliance with legal requirements are 
mandated? "however. All visitation team members must be trained, have 5 years of 
experience, be certified, and be recommended by their superintendents. Classroom 
observations and interviews are expected. 



D:24 



227 



I 
I 

1 
1 
I 
1 
I 
I 



1 
1 
I 
1 
I 
I 
I 
I 

ERIC 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

It appears that the intermediate unit has established criteria from legal sources and 
effective schools research. The staff surveys used were adapted with permission from 
The Connecticut School Effectiveness Questionnaire and the Pennsylvania Effective 
Schools Inventory. 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. What are the staff perceptions of their school in the areas of safe/orderly 
environment, clear school mission, instructional leadership/decision making, 
high expectations, opportunity to learn and student time on-task, frequent 
monitoring of student progress, and home school relations? 

2. How does a visitation team assess the school practices in the areas listed in 1? 



■ 3. What strengths/weaknesses exist in the areas of curriculum organization; 

curriculum commitment to school district goals; instructional activities; student 

■ assessment program; physical learning facilities; curriculum development; 
climate for learning; administration and supervision of curriculum, instruction, 
and staff development; classroom and library instructional resources; guiding 
student learning; and subject area programs? 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

1. Results of staff survey regarding their perceptions of the school in the areas 
of effective school correlates. 

2. Visitation team responses to guiding questions in areas listed in prior section. 
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MAIN EVALUATION METHODS USED IN THE MODEL 



What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Staff school effectiveness questionnaire (separate elementary and secondary versions); 
site visitation team; observations; interviews; examination of variety of school 
documents and student performance reports. 

MANUALS. FORMS. INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Northeastern Educational Intermediate Unit #19 Curriculum and Instruction 
Department. Curriculum, instruction, and staff development assessment 
program . Mayfield, PA: Author. 

Northeastern Educational Intermediate Unit #19 Curriculum and Instruction 
Department. Elementary school effectiveness questionnaire . Mayfield, PA: 
Author. 

Northeastern Educational Intermediate Unit #19 Curriculum and Instruction 
Department. Secondary school effectiveness questionnaire . Mayfield, PA: 
Author. 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

School effectiveness is assessed by a staff questionnaire and program review by a 
visitation team. School improvement is the primary focus. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 



Chapter 220: MilwankfiP 



Origin (Sponsor) 



Compart, for F,oW.at.iona1 Opportunity 
City of Milwaukee 



101 W EleasanLStreet, Suite ini 



Milwaukee WT 53212 



Level(s) 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

Interdistrict school integration program where students apply for and get placed at 
schools of choice. 

Mission is to "improve the quality of education and promote racial and cultural 
integration in education." 

DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Case-control design for evaluating the effectiveness of the program. 

EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Product: Student outcomes 

Input: Students, parents, and teachers 
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PUEPOSE 



Is the evaluation model formative, stimulative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

Mainly summative to this point, but formative in the sense that the program has yet 
to set its final direction. The model is intended for improvement -and knowledge 
production. 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Rigid: The procedures for applying for voluntary transfer and the laws 

governing the district 

Flexible: Student choice 

SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Compact for Educational Opportunity 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. What is the impact of the voluntary transfer program on academic growth over 
time? 

2. What impact do specific instructional strategies and curricular activities have 
on the academic growth of transfer and resident students? 

3. What impact do parental attitudes, expectations, and school-related behavior 
have on the academic growth of transfer and resident students? 

4. What are the impacts of a student's expectations, perceptions, and motivations 
on his/her academic growth? 
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5. What is the impact of the voluntary transfer program on student interpersonal 
attitudes and interactions? 

6. What is the impact of the voluntary transfer program, on student participation 
in extracurricular activities? 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

See evaluation questions above. 



MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Interviews with students, parents, and teachers 
Test scores of students in Milwaukee City Schools 



MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

1. Compact for Educational Opportunity , (May, 1989). 

2. A Proposal to Evaluate Scholastic Gains in a City-Suburban Student Transfer 
Program: The Case of Milwaukee's Chapter 220 Program . (1990). 

3. Chapter 220: Student Exchanges Between City and Suburb - The Milwaukee 
Experiences . (1978). 

4. Milwaukee's City-Suburban Interdistrict Integration Program: A Review of the 
Student Application and Assignment Process . (1990). 
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MOST DISTINCTIVE FEATURE(S) 

W it main points characterize this model? 

The effectiveness of a voluntary suburban-intercity school of choice program is 
measured in terms of student academic growth, interpersonal attitudes and 
interactions, and participation in extracurricular activities. Parental attitudes are 
also evaluated. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 



Origin (Sponsor) 



Omaha Pnhlif! Schools 



3215 CiimingLStEfifit 



Omaha, NE fiaiai=2Q24 



Level(s) 



AH fftvftry building on subject, matter Cydfi) 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

Specific subject matter programs are evaluated by building on a 6-year cycle (i.e., 
math in first year, language arts and English core in second year, etc.). 

M The Omaha Public Schools' Program Analysis and Evaluation Plan is a written 
appraisal designed to determine the effectiveness of the program, given available 
resources and district priorities. Specific criteria or standards of effectiveness are 
basic components used to evaluate each program and its support services." 

The model was originally adapted from the Wisconsin School Evaluation Consortium 
model (Program Evaluation Through Self study ). 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Program effectiveness is assessed in three areas: 

1. Program Intent - including review of the intent of the school system in relation 
to the program that is being evaluated; determination of the degree to which 
agreement has been reached in the school district relative to the philosophy, 
purpose, and expected learning outcomes for the program; and evaluation of 
possibilities for revising the intent of the program in future revisions. 

2. Evidence of Program Success - including surveys of opinions from staff, 
parents, students, graduates of the school district, employers, and university 
staff; data about student performance; information about instruction and 
school events gathered by staff. 
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Evaluation and Judgment - "School and district-wide committees will weigh « 
the evidence against the standards set for the program and will identify g 
strengths, needs, and suggestions for improvement. Standards used will be 
program intent (philosophy, purpose, and expected learning outcomes) and 
Best Educational Practice Standards (BEPS) selected for the Program Analysis 
and Evaluation Process." 



EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Qualitative and quantitative data are used (". . . Facts, figures, descriptions, and 
carefully collected opinions . . ."). 

Evaluation is intended to be process oriented, with additional input and product 
components. 



PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

"Articulation and continual evaluation are necessary for educational systems to 
determine the extent to which goals are being accomplished. 'Educational excellence' 
continues to be a priority for the school district. The integration of program analysis 
with selection of textbooks, development of supportive curricular materials and 
ongoing identification of district priorities insures 'education excellence' for the School 
District of Omaha." 

Intended for improvement and accountability; is formative and summative. 
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ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Quite rigid: Step-by-step process is defined; some latitude in how program strengths, 
program needs, suggestions for school improvement plans are identified. 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

The district has adopted the "Best Educational Practice Standards." (Each program 
evaluation is to include an assessment of extent to which the program is meeting 
each cluster of standards.) 

Program intent standards (philosophy, purpose, expected learning outcomes) are 
established by district committees. 



MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. To what extent are the goals being accomplished in a specific subject 
discipline? 

2. To what extent are the program intent standards and the "Best Educational 
Practice Standards" being met? 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

1. Program Intent - review of intent of program, philosophy, purpose, expected 
outcomes, recommendations for future revisions. 

2. Evidence of Program Success - opinions of staff, parents, students, graduates 
of the school district, employers, and university staff; student performance 
data; information about instruction and school events gathered by staff. 
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3. Evaluation and Judgment - school and districtwide committees will weigh the 
evidence against the standards set for the program and identify strengths, 
needs, and suggestions for improvement. 



MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Self-Study, District/Building Profile Information 

Sources of information are: 

Districtwide surveys completed in September-October 
Information collected from district databases and records 
Information collected from staff in a school 

Visitation team is used, 

MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

School District of Omaha. Best educational practice standards . Omaha, NE: 
Author. 

School District of Omaha. Six-year cycle . Omaha, NE: Author. 

South High School (1989-90). School summary report: Program analysis and 
evaluation (art) . (Sample of report done by each building in specified subject 
disciplines each year. General directions are included.) 
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MOST DISTINCTIVE FEATURE(S) 



What main points characterize this model? 

The Omaha Public Schools (and its individual schools) evaluate instructional 
programs on a 6-year cycle. Program intent, evidence of program success, and 
evaluation and judgment are examined. The process involves determination of the 
degree to which the district's "Best Educational Practice Standards" and the 
program's standards of intent are met. A variety of indicators are used. 



D:35 



SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 



School Tmprnvftmpnt 



Origin (Sponsor) 



Portland Pnhlir Schools 



5Q3 N DiYnn Street 



p.n. Rot am? 



Portland. OR 379.9.7 



All fhy individual school, cluster, an d district) 



Level(s) 
APPROACH 



What is the model's pervasive orientation? What is the essence of the 
luation model? 



Individual school improvement plans are developed and revised through a continuous 
process of needs assessment, evaluation of instructional programs, compliance with 
state requirements, implementation of improvement, staff development, reporting test 
results to the community, reporting to the Department of Education, and revision of 
plan. Extensive data are kept by the district on disaggregated test results "to 
support data-based decision making for improvement of instruction and instructional 
programs/' Data are reported by each school, school cluster, and district. 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

(See prior section.) 



EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Context: Needs assessment 

Input: Meeting of state requirements 

Process: Ongoing school improvement efforts 

Product: Test results, interview findings 
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Much emphasis on quantitative data, but some qualitative evaluation also is 
included. 

PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

This model is both formative and summative. It is intended for improvement and 
accountability. 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Individual schools have flexibility in the development of their school improvement 
plans. However, there are steps they must follow: needs assessment, mission 
statement, goal statement, goal support activities, process to support change, 
evaluation. Fall to spring gains in reading, mathematics, and language usage are to 
be included in the report. 

SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Legal requirements set by the state. 

The format for planning and reporting is set by the district. Standards appear to be 
set both by central office personnel and by individual schools. 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. Are the school system and its individual schools meeting the legal 
requirements set forth by the state? 

2. Is student achievement improving in reading, mathematics, language usage, 
and other areas? 
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3. What are the individual school improvement goals, subgoals, and plans for 
accomplishing these goals and subgoals? (Areas include educational 
effectiveness, communication, management, and fiscal effectiveness.) 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

1. Checklist for legal requirements 

2. Test results (Portland Achievement Level Tests) 

3. Individual school improvement plans 

MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

1. District data collection (demographics, test results) 

2. Self-study (against legal requirements) 

3. External standardization visit (to confirm self-study results) 

4. Review of school improvement plans; observations and interviews in schools 



MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Beck, J. W. (1990). School improvement plan forma l Portland, OR: Portland Public 



Hathaway, W. (1990). Materials for standardization visit . Portland, OR: Portland 
Public Schools. 

Portland Public Schools. (1990, September). Progress report, 1989-90: Goals 
and objectives . Portland, OR: Author. 

Portland Public Schools. (1991, May). Some key facts about student performance in 
the Portland Public Schools based upon our high standards and accurate 
measures . Portland, OR: Author. (10 year summary report). 

Research and Evaluation Department (Walter Hathaway, Director). (1990). 1989-90 
test results: Administrative reports . Portland, OR: Portland Public Schools. 



Schools. 




Samples of Individual School Instructional Improvement Plans: 
Ball School (PK-5), 1989-90 
Clarendon School (K-5), 1987-88 

Vernon ECEC, School Improvement Plan (Revised 1990-91) 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

Much disaggregated and aggregated data are used to guide instructional decisions. 
A school improvement process is ongoing in individual schools. An external 
Standardization Visitation Team confirms self-study findings. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source Effective Schools AiiditlAjhrisory Erocass; Tnrliratnrs of 

Srhonl Ef fectiveness fHanrlhnok^ 

Origin (Sponsor) Prince Ge orges County Pnhhr, Schools 

14901 School Lane 

Upper Marlboro, MP 90779 

Level(s) Middle & High Schools (This document says for 

miflHIe/hi gh schools; appears appropriate for elementary, 
too, anH all schools are said tn he involved in this ) 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? What is the most distinguishing feature? 

The audit/advisory process includes compilation and analysis of data that support 
student achievement and participation indicators; completion of a staff survey 
focusing on school climate; writing of a preliminary staff report; on-site visitation by 
qn Audit/Advisory Team including interviews of School Improvement Team and other 
staff members; and an exit report with school's performance on major "indicators of 
effectiveness," commendations for exemplary performance, and recommendations for 
future revisions of the school improvement plan. 

DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

"The initial Effective Schools Audit Process was designed to give schools input as to 
the degree to which the School Improvement Plan was being implemented. Emphasis 
. . . was on staff awareness and involvement, and the extent of implementation of the 
plan." 

In its fifth year, "... The Audit Process has been updated to reflect the present 
advanced level of local school implementation. This revised Audit Process focuses on 
the indicators of school effectiveness, and requests schools to provide evidence of 
school effectiveness in terms of (1) student outcomes, (2) the presence/absence of 
correlates representing effective schools, and (3) the degree to which school based 
management supports the school improvement plan." 
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EVALUATION TYPE(S) 



Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Product: Student outcomes in achievement and participation 

Context: Survey of school climate 

Process: Ongoing school improvement efforts 

Input: Fulfilling effective schools correlates; also process and context 

Quantitative and qualitative information included. 



PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

"The School Improvement Process . . . has as its goal teaching for learning to achieve 
quality and equity for all students within the school system." 

"The intent of the revised audit process is to: 

1. Assist schools in examining the degree to which their school improvement 
efforts have established quality and equity within the school. 

2. Identify ways in which the Effective Schools Process can help the school reach 
its goals. 

3. Provide recommendations to the School Staff and the School Improvement 
Team/School Planning and Management Team for possible adjustments in the 
implementation process. 

4. Serve as a Maryland School Performance Program." 
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^LOWANCE FOR IND 



UAL APPROACHES 



To what extent is th ;del rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

June Criterion Referenced Test (CRT) performance levels are specified (70 percent) 
for all grade levels. Some data are to be reported (e.g., attendance rate, graduation 
rate, etc.). Evidence of presence of effective school correlates can be documented in 
a variety of ways. 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Not stated. Much of this comes from Effective Schools Research, however. 



MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. To what extent are achievement indicators and participation indicators reached 
in this school? 

2. To what extent are effective schools correlates present or absent in this school? 

3. T 'hat extent is school-based management used in this school? 

4. H, do the studem outcomes compare with the Standards for MSPP Data- 
Based Areas? 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

1. Indicators of school effectiveness (Student Outcomes) 

a. Achievement indicators - percentages of students at grade level 
in reading and mathematics whose June CRT results in reading 
i \ mathematics exceed 70 percent, promoted « elementary) 
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b. Achievement indicators - percent of students enrolled in Pre- 
Algebra and Algebra I in grades 7 & 8, in foreign language, 
receiving awards or recognition for academic growth, promoted, 
exceeding 70 percent on June CRT. 

c. Achievement indicators - percent of students whose June CRT 
results exceed 70 percent, students enrolled in higher level 
courses over a 3 year period, students applying to postsecondary 
institutions, graduates receiving honors and awards, SAT results 
over 3 year period, students promoted, students passing 
functional test on first administration, students passing 
functional test by graduation. 

d. Participation indicators - percentages of student attendance, 
teacher attendance, suspensions, expulsions, students receiving 
awards for making contributions to school and/or community 
(elementary and secondary). 

e. Participation indicators - percent of students with a GPA of 2.0 
or better, student participation in extracurricular activities, 
participation in PSAT and SAT testing opportunities, student 
dropouts (secondary only). 

Evidence of presence or absence of effective schools correlates 



a. 


Strong instructional leadership 


b. 


Clear and focused mission 


c. 


Opportunity to learn and time on learning 


d. 


Frequent monitoring of student programs 


e. 


Positive school climate 


f. 


Positive home-school relations 



Evidence of effective school-based management 

a. Plan for resources (time, staff, money, materials) to support 
school improvement goals 

b. Evidence of improved student outcomes 
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c. Evidence that staff participated in decisions about allocation of 
resources 

d. Monitoring plan for expenditures 
4. Outcome data against MSPP Standards 

MAIN EVALUATION METHODS USED IN THE MODEL 



i 

i 
i 

i 

What are the core data collection methods, e.g., state-mandated information B 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management I 
team, teachers), examination of financial books? ™ 

Student outcomes report (student achievement and participation data); school profile H 
packet; survey (instructional staff and support staff); preliminary staff report, site * 
visitation team, exit report. 

i 

i 



MANUALS, FORMS. INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Prince George's County Public Schools. (1991, January). Effective schools Q 
audit/advisory process: Indicators of school effectiveness (Handbook: Middle and 
high schools) . Prince George's County , MD: Author. 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 



A study of student achievement and participation outcomes, climate survey results, 
indicators of presence or absence of effective schools correlates, and evidence of 
effective site-based management is conducted. An audit team confirms the self-study 
and identifies areas in which school effectiveness is indicated; areas of exemplary 
performance; and suggestions for School Improvement Plan revisions. Follow-up is 
decided upon by the school staff and the Assistant Superintendent. j| 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 

Origin (Sponsor) Rkhland Cniinty Schno] District Onp 

lfilfijakhland Street 

Columbia, SC 

Level(s) All; individual school 

APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

Evaluation appears to be done at the individual building level. Measures of opinion 
about dimensions of the school are collected on surveys to teachers, classified 
personnel, students, and parents. Outcomes in the areas of student achievement, 
student attendance, teacher attendance, and student dropout rates (middle and high 
school) are assessed in the "School Effectiveness Outcomes Assessment." 

DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

A comprehensive summary of a school's performance on specific outcome measures 
is compiled. Performance data are summarized in the areas of student achievement, 
student attendance, teacher attendance, and student dropout rates (middle and high 
schools). 

In addition, attitudes of teachers, classified personnel, students, and teachers toward 
dimensions of the school are measured with surveys. 

EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Product: Outcomes areas of student achievement, student attendance, teacher 

attendance, student dropout rates 
Context: From survey information 
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Quantitative data are primarily used. Some qualitative material is addressed in 
surveys. 

PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

"The School Effectiveness Outcomes Assessment report is designed to provide a 
comprehensive summary of a school's performance on available outcome measures." 
Used for summative judgments, accountability, and improvement. 

Questionnaires to examine percent of positive responses of teachers, classified 
personnel, students, and parents appear to be intended to measure positive attitude 
toward different dimensions of the school. 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

The model is rigid. Test instruments and criteria to meet status levels are defined. 
However, ". . . A school's group assignment under the South Carolina School grouping 
system is taken into consideration for setting standards." 

SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Not stated 
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MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. Does student achievement meet or exceed the minimum standard, or has 
improvement been shown since the prior year? 

2. Does student attendance meet or exceed the minimum standard, or has 
increase been shown since the prior year? 

3. Does teacher attendance meet or exceed the minimum standard, or has 
increase been shown since the prior year? 

4. Has the student dropout rate (middle and high school) met standard, or has 
rate lowered since the prior year? 

5. How positive are teacher, classified personnel, student, and parent opinions of 
(a) principal leadership, (b) student enthusiasm/academic emphasis, (c) high 
expectations, (d) school climate, (e) monitoring progress, (f) home-school 
relations? 



EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

1. Scale scores for school groups (grade levels) by subject area 

2. Student attendance percentage (based on State Quality Assessment method 
(135 day report ADA/ADM) 

3. Teacher attendance percentage . 

4. Dropout percentage rate calculated using State Quality Assessment method 

5. Percent of positive responses for school and district on questionnaire items and 
clusters of items 

MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g M state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 
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Performance achievement scores; school profile information on student and teacher 
attendance, and dropout r surveys to teachers, classified personnel, students, and 
parents. 



MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Richland County School District One. (1990, October 26). School effectiveness 
outcomes assessment . Columbia, SC: Author. 

Richland County School District One. (1990). Survey summary results for example 
schools . Columbia, SC Author. (H. S. Classified Personnel Results, 
Elementary Student Survey Results, Middle School Parent Survey Results, 
Teacher Survey Results. 

MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

Outcome indicators in student achievement, student attendance, teacher attendance, 
and dropout rate are compiled and compared against defined standards. Opinion 
surveys from teachers, classified personnel, students, and parents assess degree of 
positive response to several school dimensions. 
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Title or Source 
Origin (Sponsor) 

Level(s) 
APPROACH 



SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 

Site-Based Management Program 



Rochester City Srhnol District 



131 W Broad StrPPt 



Rochester, NY 14fi14 



Elementary, middle, and fiennnHary 



What is the model's pervasive orientation? What is the essence of the 
evaluation model? What is the most distinguishing feature? 

School Based Planning Steering Committee deals with issues of instruction, student 
performance, and school environment/improvement. All decisions are binding if they 
are within the authority of the school. 

'Team must prepare and implement an ANNUAL SCHOOL IMPROVEMENT PLAN 
for Improved Student Performance." 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

"Evaluation - describes the measures which will be used to determine outcomes based 
on goals/' 



EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Input: From teachers, principals, parents, students, etc. 

Outcome: Quality education 
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PURPOSE 



Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

Formative: Setting goals for next year 

Summative: Evaluation of past year(s) improvement plan(s) 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Rigid: Steering committee must have constituency consensus or proposal does 

not pass. 

Flexible: Site planning is able to set specific requirements for each particular 
school 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Rochester School Board along with the District Superintendent 



MAIN EVALUATION QUESTIONS 

What are t he main questions to be answered by the evaluation? 

1. Doe; the evaluation report accomplish all of last year's school improvement 
plan goals? 

2. Are the needs of the particular school being met by this year's school 
improvement plan? 
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EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

1. Evaluation report 

2. School improvement plan 

MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observation, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Site-based planning team conducts a self-study each year and fills out an evaluation 
report. 

MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

1. Annual Planning Cycle 

2. Rochester City School District Site-Based Management Program Fact Sheet 

3. School Accountability 



MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

Site-based planning teams: teachers, administrators, parents, and students work 
together to improve their schools. 
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SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 



Title or Source 
Origin (Sponsor) 

Level(s) 



Saginaw SllCCfiSSfil] Schools P rojert (S3E3 

vSr.hool District of the City nf Saginaw 

SOP Millard 

Saginaw, MT 4ftfi07 

Elementary; secondary (district project, imp lemented tn 
large extent in individual buildings^ 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

This project was begun in 1981-82 "to fit together all of the pieces of the puzzle" (i.e., 
assertive discipline, instructional theory into practice, programs for gifted and 
talented students, programs for academically disadvantaged students, zero base 
budgeting, etc.). The project is defined as a ". . . Comprehensive school improvement 
model." 

"The total focus of the Saginaw Successful Schools Plan will be to consolidate all 
important aspects of education and *bcus all such efforts on the contribution that 
these make to improving instruction for children ... all support systems within the 
district should form an infra-structure which contributes to instruction." 

Ongoing evaluation is implied in the continuous revision of school improvement plans 
based upon needs assessments. 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

1. Assess the extent to which we are adequately and properly addressing each 
factor or variable and thereby identify our needs. 

2. Identify the resources (funds, personnel, time, program) to address each need. 

3. Provide for formative and summative evaluation of progress. 
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Effectiveness of this project is being studied primarily by analyzing staff opinion and 
student performance (test scores). 



EVALUATION TYPE(S) 



Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Mostly quantitative information is collected. Context (needs assessments), product, 
and process are addressed. 
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PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

Formative and summative; intended primarily for improvement as well as 
accountability. 

"We wish to bring about the following: equal educational opportunity for all students; 
a unified K-12 curriculum; an expanded basic skills curriculum that includes reading, 
mathematics, science, and social studies; consistent grade to grade expectations and 
appropriate objective referenced measures of them; and, policies of the Board of 
Education and administration that are constructed so that they focus all efforts 
toward improved instruction in light of the research base." 



ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

This model is flexible. Schools set their own goals and objectives for the most part. 
Evaluation of success can be done in a variety of ways. 
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SOURCE OF EVALUATION STANDARDS 



Who establishes evaluation standards and criteria within the model? 

A 5-year plan developed by a 23-member task force of volunteers within the district. 
Members were "representative" of the district 

Evaluation of individual school plans appears to be done by that school's staff and 
central office staff. 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. What is staff involvement in S3P project? 

2. What areas need to be addressed to improve the schools in Saginaw? 

3. Are students benefiting from the S3P project? 

4. What are individual schools' plans for improving education? 

EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

1. Number and percent of staff completing S3P Staff Commitment Survey (by 
level) 

2. School Improvemen Survey findings (communications/teamwork; 
administrator/teacher relations; school effectiveness; instructional 
effectiveness) 

3. Priority need index of items from survey above (least need possible to greatest 
need possible; improvement shown when opinions move to "least need possible" 
end of scale) 

4. Student achievement test scores 

5. Existence of individual school improvement plans 
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MAIN EVALUATION METHODS USED IN THE MODEL 



What are the core data collection methods, e.g., state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

School Improvement Survey (1-8 scale; items in areas of Teamwork and 
communication; administrator-teacher relations; school effectiveness; instructional 
effectiveness and levels of needs identified). Profile of level of commitment and 
involvement in each building by elementary, middle school, high school, special sites, 
and overall district. District profile: individual annual program plans; use of data 
to support decision making. Test Scores (MEAP, Norm-Referenced CAT) 



What is available for users and for others interested in reviewing the model? 

School District of the City of Saginaw. (1986). Accent on achievement: Past, 
present, future . (Annual Report of the Superintendent, 1985-86.) 

School District of the City of Saginaw. (1985). A description of the Saginaw 
Successful Schools Project, 



School District of the City of Saginaw. (1988). Evidence that S3P is working . 
(List and overhead information). 

School District of the City of Saginaw. (1985) Features of S3P . 

School District of the City of Saginaw. (1983). Notes on the history and development 
of the Saginaw Successful Schools Project . 

School District of the City of Saginaw. (1987, August), Saginaw successful 
schools project - Phase II: An outline for the next five years . 

School improvement plan synopsis: Saginaw Successful Schools Project . 

Saginaw's successful schools project: A plan for program improvement . (1982 draft) 



MANUALS, FORMS. INSTRUCTIONS 
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MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

Evaluation of this project is primarily focused on staff perceptions of needs in four 
key areas and student achievement gains. School improvement has been a 
continuous process in the individual buildings of this district since 1982. 
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Title or Source 

Origin (Sponsor) 

Level(s) 
APPROACH 



SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 

The Spring Branch Modfil fbr Increasing School 

F.fferf.iveness Through More Campus-Rafted Derision 

Making 

Spring Branch Independent School District 

955 CamphfiU Rnad 

Houston, TY 77094. 

All levels (individual schools^ 



What is the model's pervasive orientation? What is the essence of the 
evaluation model? 



The school improvement plan is "a process at the application level, not a series of 
events." The process is based on effective schools research and correlates and 
involves much decision making at the individual campus level. ". . . The central office 
is at the bottom of the chart undergirding the work of the most strategic unit within 
the district— the school." 



Central office, collaborative, and campus responsibilities are defined in the areas of 
personnel management, human resource development, budget, organization, 
curriculum, monitoring, climate marketing. 

Three types of schools are contrasted on a continuum of least to most desired: 
custodial school, effective school, restructured school. 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

Evaluation is a three-stage process: 

1. Preparation Stage : Development of staff awareness of effective schools; 
formation of Campus Improvement Team (CIT); clarification of 
roles/responsibilities of CIT. 

2. Diagnosis Stage : Examine progress of district mission/adjust current Campus 
Improvement Plan (CIP); collect/examine/study four data sources. 
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3. Planning/Implementation Stage : Develop/create objectives/ activities; seek CIT 
consensus on CIP; solicit endorsement of CIP; implement CIP. 

Data-driven, goal-oriented, objective-based processes for improvement ere the focus. 



EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? Mixture? Qualitative or quantitative? Other? 

Product (student achievement), Process, Context (needs/attitude assessments) are 
included. Qualitative and quantitative data are used for accountability and to make 
decisions about future plans. 

PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

This model is both formative and summative. It is primarily used for improvement, 
but accountability is built into it (outcome measures). 

Its stated purposes are (1) to increase the decisions about teaching and learning at 
each campus location; (2) to better align the work of central staff personnel so as to 
effectively support each campus leadership team; (3) to reduce the number of central 
office staff; and (4) to share the accountability for the performance of each student 
with the campus leadership teams and the district's central staff. 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

Three kinds of data are collected in the needs assessment process associated with 
effective school planning at each site: student achievement data, archival or 
historical data, survey data. These are specified. Additional data can be identified 
by the CIT. There is a mixture of rigidity and flexibility in the model. 
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Who establishes evaluation standards and criteria within the model? 

Each Campus Improvement Team establishes its goals and objectives to fit effective 
schools correlates. At least part of the CIP must tie in with the district key 
improvement priorities. 



MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. Are all students achieving? 

2. What are the characteristics of the student and community population? 

3. How do staff members feel about their school? 

4. Other questions identified by specific schools. 

Also, will a specific activity in the CIP become practice rather than an event? Is the 
activity important? Is the proposed activity consistent with effective schools research, 
principles, and practices? 

EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

Student achievement data (disaggregated by socioeconomic status): norm-referenced 
tests, criterion-referenced tests, student retentions/placements by grade level 

Disaggregated historical or archival data for analysis: suspensions, expulsions, 
dropouts, and attendance; student activities such as organizations, clubs, athletics; 
staff characteristics (professional growth, teacher evaluation results, faculty 
attendance); and parent involvement 

Results of annual Survey of Effective School Correlates 

Informal Data: Additional variables at individual campus schools (e.g., student or 
parent surveys and/or identification of common perceptions/problems/issues present 
at school). 
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MAIN EVALUATION METHODS USED IN THE MODEL 



What are the core data collection methods, e.g*, state-mandated information 
system, district profile, self-study, site visit team, community surveys, 
observations, interviews (board, superintendent, principal, management 
team, teachers), examination of financial books? 

Development of campus improvement plan, compilation of data (student achievement, 
archival or historical data, survey data, other data specified by individual CIT), 
review of CIP progress. 

MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Guthrie, H., Mathews, G. S., & Wells, B. (1990, December). Increasing school 
effectiveness through more campus-based decision making: Restructuring our 
schools. Instructional Leader (Texas Elementary Principals and Supervisors 
Association), 3(6), 1-8. 

Guthrie, H., Mathews, G. S., Youdan, D., Sumner, M., & Wells, B. (1989, June). 

Campus improvement planning: A systematic team approach. Instructional 
Leader (Texas Elementary Principals and Supervisors Association), 2(3), 1-9. 

Guthrie, H., Mathews, G. S., Youdan, D., Sumner, M., & Wells, B. (1989, August). 
Part II: Campus improvement planning: A systematic team approach. 
Instructional Leader, 2(4), 1-10. 

Spring Branch Independent School District. (1991, Spring). Survey of effective 
school correlates . (Adapted from the Connecticut School Questionnaire). 
Houston, TX: Author. 

Spring Branch Independent School District. Together in Excellence (packet) . 
Houston, TX: Author. (Includes charts, outline for data collection, annual 
planning cycle, description of the model, responsibilities of central office, 
central office and campus, and campus activities in various school functions, 
"A Developmental Staging Plan for Restructuring a School.") 
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MOST DISTINCTIVE FEATURE(S) 

What main points characterize this model? 

Ongoing campus improvement efforts (cycle of preparing, diagnosing, and 
planning/implementation). Campus Improvement Teams develop Campus 
Improvement Plans for their schools. The plans address the district key improvement 
priorities as well as other areas identified by individual teams. 
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Title or Source 
Origin (Sponsor) 



SCHOOL EVALUATION MODEL 
PROFILE DATA FORM 

Program Evaluation Through SoILStlldy 

Wismnsin School Evaluation Consortium (SEC) 

University of Wisconsin-Madison 

427 Education K1d g 



inn Baacom Mall 



Madison WT 53ZQfi 



Level(s) 



K-12; pnhlir, private, parochial (often whole district) 



APPROACH 

What is the model's pervasive orientation? What is the essence of the 
evaluation model? 

Guidelines for carrying out a self-study are provided; local districts are advised to 
tailor their own evaluation processes to their districts. Strengths/weaknesses and 
recommendations for improvement/change result from the self-assessment. It is not 
intended for programs in formative stages. "The SEC process is an example of a 
change process and conducting self studies is a step in promoting evolutionary change 
in schools." 



DEFINITION OF EVALUATION USED IN THE MODEL 

How does the model implicitly or explicitly define evaluation? 

"Evaluation as a change process . . . deciding to use an evaluation process indicates 
that an organization is committed to change." ". . . Evaluation produces better results 
when it is locally designed and owned." However, all shox\ld contain five features: 
(1) identified leadership, (2) a long-range plan, (3) identified self-study procedures, 
(4) carefully articulated roles; and (5) coordination of SEC activities with other 
district/school functions. Four purposes of evaluation: (a) measurement, (b) 
judgment, (c) description; (d) addressing informational needs. "Conducting an 
evaluation involves critically examining a program for its improvement. The process 
results in providing information about the program and promotes communication 
among those who care about, are interested in, or affected by the program of study." 
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EVALUATION TYPE(S) 

Is model essentially Context, Input, Process, Product? Interactive or 
preordinate? mixture? Qualitative or quantitative? Other? 

1. Qualitative and quantitative data collection are encouraged. 

2. Evaluation is process oriented and interactive in nature. 

PURPOSE 

Is the evaluation model formative, summative, or both? Is the model 
intended for improvement, accountability, knowledge production, 
dissemination of good program practices? 

Intended for improvement- formative and summative (but not intended for programs 
in formative stages; is intended primarily for in-place programs). 

ALLOWANCE FOR INDIVIDUAL APPROACHES 

To what extent is the model rigid or flexible in requiring specified criteria, 
procedures, instruments, evaluators? 

This model is very flexible. People/organizations using the model are to design their 
own models within a broad framework defined in this manual. 



SOURCE OF EVALUATION STANDARDS 

Who establishes evaluation standards and criteria within the model? 

Standards and criteria are locally determined. Professional organizations and state 
education departments are suggested as sources to use. 

MAIN EVALUATION QUESTIONS 

What are the main questions to be answered by the evaluation? 

1. What is the description of the program (to establish program intent or describe 
select program factors)? 
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2. What is the focus of the self-study? 

3. To what extent does this program meet or exceed chosen standards? 

4. What actions should follow the self-study to improve the program? 

EVALUATION VARIABLES 

What indicators need to be assessed to address the evaluation questions? 

There are six steps: 

1. Program description 

2. Purposes and selection of key questions 

3. Gathering information 

4. Analyzing information, developing conclusions, and communicating results 

5. Auditing and maximizing results 

6. Managing evaluation (in each step and overlaps) 

MAIN EVALUATION METHODS USED IN THE MODEL 

What are the core data collection methods, e.g., state-mandated information system, 
district profile, self-study, site visit team, community surveys, observations, 
interviews (board, superintendent, principal, management team, teachers), 
examination of financial books? 

Self-study is the primary method used. "Procedures must be flexible so that program 
differences and reasons for conducting evaluation itself can be accommodated." 

MANUALS, FORMS, INSTRUCTIONS 

What is available for users and for others interested in reviewing the model? 

Blasczyk, J., & Dirkx, J. (1989). Program evaluation through self study . Madison, 
WI: School Evaluation Consortium (University of Wisconsin - Madison). 

Training materials on the managerial role of steering committees, alternatives to 
using the best educational practice standards, narrowing the scope of self-studies, 
limited and purposeful data collection based on key questions, alternatives to surveys, 
data analysis, and implementation of self-study findings are in manual and can be 
purchased separately as well. 

D:64 



267 



The school evaluation consortium: A voluntary partnership for quality education 
programs . Pamphlet. 

Two other manuals are referred to on second cycle planning. 



MOST DISTINCTIVE FEATURE(S) 
What main points characterize this model? 

The purpose of this model is to assist SEC members, especially those who serve on 
program committees, when conductnig program evaluations through self-study. 
"Group is involved in an evolving process." "Evaluation is a change process." A long- 
range evaluation plan is established and carried out; an action plan results after 
recommendations are audited; assistance is available through SEC consultants and 
others (by choice of district). Each consortium member has an assigned field 
consultant. Numerous training events are conducted for members. 
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